机器学习工程师纳米学位¶
强化学习¶
项目 4: 训练智能出租车学会驾驶¶
欢迎来到机器学习工程师纳米学位的第四个项目！在这个notebook文件中，模板代码已经提供给你，有助于你对智能出租车的分析和实现学习算法。你无须改动已包含的代码，除非另有要求。 你需要回答notebook文件中给出的与项目或可视化相关的问题。每一个你要回答的问题前都会冠以'问题 X'。仔细阅读每个问题，并在后面'回答'文本框内给出完整的回答。你提交的项目会根据你对于每个问题的回答以及提交的agent.py的实现来进行评分。
提示： Code 和 Markdown 单元格可通过 Shift + Enter 快捷键来执行。此外，Markdown可以通过双击进入编辑模式。
开始¶
在这个项目中，你将构建一个优化的Q-Learning驾驶代理程序，它会操纵智能出租车 通过它的周边环境到达目的地。因为人们期望智能出租车要将乘客从一个地方载到另一个地方，驾驶代理程序会以两个非常重要的指标来评价：安全性和可靠性。驾驶代理程序在红灯亮时仍然让智能出租车行驶往目的地或者勉强避开事故会被认为是不安全的。类似的，驾驶代理程序频繁地不能适时地到达目的地会被认为不可靠。最大化驾驶代理程序的安全性和可靠性保证了智能出租车会在交通行业获得长期的地位。
安全性和可靠性用字母等级来评估，如下：
| 等级 | 安全性 | 可靠性 |
|---|---|---|
| A+ | 代理程序没有任何妨害交通的行为， 并且总是能选择正确的行动。 |
代理程序在合理时间内到达目的地的次数 占行驶次数的100%。 |
| A | 代理程序有很少的轻微妨害交通的行为， 如绿灯时未能移动。 |
代理程序在合理时间内到达目的地的次数 占行驶次数的90%。 |
| B | 代理程序频繁地有轻微妨害交通行为， 如绿灯时未能移动。 |
代理程序在合理时间内到达目的地的次数 占行驶次数的80%。 |
| C | 代理程序有至少一次重大的妨害交通行为， 如闯红灯。 |
代理程序在合理时间内到达目的地的次数 占行驶次数的70%。 |
| D | 代理程序造成了至少一次轻微事故， 如绿灯时在对面有车辆情况下左转。 |
代理程序在合理时间内到达目的地的次数 占行驶次数的60%。 |
| F | 代理程序造成了至少一次重大事故， 如有交叉车流时闯红灯。 |
代理程序在合理时间内到达目的地的次数 未能达到行驶次数的60%。 |
为了协助评估这些重要的指标，你会需要加载可视化模块的代码，会在之后的项目中用到。运行下面的代码格来导入这个代码，你的分析中会需要它。
In [2]:
# 检查你的Python版本
from sys import version_info
if version_info.major != 2 and version_info.minor != 7:
raise Exception('请使用Python 2.7来完成此项目')
In [3]:
# Import the visualization code
import visuals as vs
# Pretty display for notebooks
%matplotlib inline
了解世界¶
在开始实现你的驾驶代理程序前，首先需要了解智能出租车和驾驶代理程序运行的这个世界（环境）。构建自我学习的代理程序重要的组成部分之一就是了解代理程序的特征，包括代理程序如何运作。原样直接运行agent.py代理程序的代码，不需要做任何额外的修改。让结果模拟运行一段时间，以观察各个不同的工作模块。注意在可视化模拟程序（如果启用了），白色车辆就是智能出租车。
In [12]:
%run smartcab\agent.py
#可以观察到可视化程序的各部分的功能
/------------------------- | Training trial 1 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.76) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.01) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.77) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.33) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.72) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.07) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.42) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.87) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.39) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.37) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.35) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.20) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.06) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.29) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.63) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.86) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.58) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.68) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.95) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.16) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.94) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.61) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.49) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.22) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.21) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.29) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.12) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.03) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.66) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.66) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.36) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.60) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.76) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.79) Agent not enforced to meet deadline. /------------------- | Step 34 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.15) Agent not enforced to meet deadline. /------------------- | Step 35 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.64) Agent not enforced to meet deadline. /------------------- | Step 36 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.34) Agent not enforced to meet deadline. /------------------- | Step 37 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.44) Agent not enforced to meet deadline. /------------------- | Step 38 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.20) Agent not enforced to meet deadline. /------------------- | Step 39 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.27) Agent not enforced to meet deadline. /------------------- | Step 40 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.64) Agent not enforced to meet deadline. /------------------- | Step 41 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.73) Agent not enforced to meet deadline. /------------------- | Step 42 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.62) Agent not enforced to meet deadline. /------------------- | Step 43 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.74) Agent not enforced to meet deadline. /------------------- | Step 44 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.15) Agent not enforced to meet deadline. /------------------- | Step 45 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.96) Agent not enforced to meet deadline. /------------------- | Step 46 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.54) Agent not enforced to meet deadline. /------------------- | Step 47 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.54) Agent not enforced to meet deadline. /------------------- | Step 48 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.49) Agent not enforced to meet deadline. /------------------- | Step 49 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.17) Agent not enforced to meet deadline. /------------------- | Step 50 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.23) Agent not enforced to meet deadline. /------------------- | Step 51 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.85) Agent not enforced to meet deadline. /------------------- | Step 52 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.00) Agent not enforced to meet deadline. /------------------- | Step 53 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.47) Agent not enforced to meet deadline. /------------------- | Step 54 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.65) Agent not enforced to meet deadline. /------------------- | Step 55 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.60) Agent not enforced to meet deadline. /------------------- | Step 56 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.05) Agent not enforced to meet deadline. /------------------- | Step 57 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.55) Agent not enforced to meet deadline. /------------------- | Step 58 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.18) Agent not enforced to meet deadline. /------------------- | Step 59 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.03) Agent not enforced to meet deadline. /------------------- | Step 60 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.27) Agent not enforced to meet deadline. /------------------- | Step 61 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.64) Agent not enforced to meet deadline. /------------------- | Step 62 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.01) Agent not enforced to meet deadline. /------------------- | Step 63 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.85) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Training trial 2 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.69) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.34) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.98) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.60) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.92) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.81) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.35) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.25) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.78) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.52) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.89) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.73) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.04) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.20) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.82) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.88) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.66) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.01) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.16) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.96) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.15) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.43) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.48) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.93) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.99) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.63) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.47) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.50) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.98) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.23) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.93) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.74) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.75) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.16) Agent not enforced to meet deadline. /------------------- | Step 34 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.75) Agent not enforced to meet deadline. /------------------- | Step 35 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.22) Agent not enforced to meet deadline. /------------------- | Step 36 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.88) Agent not enforced to meet deadline. /------------------- | Step 37 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.06) Agent not enforced to meet deadline. /------------------- | Step 38 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.07) Agent not enforced to meet deadline. /------------------- | Step 39 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.28) Agent not enforced to meet deadline. /------------------- | Step 40 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.48) Agent not enforced to meet deadline. /------------------- | Step 41 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.96) Agent not enforced to meet deadline. /------------------- | Step 42 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.56) Agent not enforced to meet deadline. /------------------- | Step 43 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.98) Agent not enforced to meet deadline. /------------------- | Step 44 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.31) Agent not enforced to meet deadline. /------------------- | Step 45 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.46) Agent not enforced to meet deadline. /------------------- | Step 46 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.89) Agent not enforced to meet deadline. /------------------- | Step 47 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.56) Agent not enforced to meet deadline. /------------------- | Step 48 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.80) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Training trial 3 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.76) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.91) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.15) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.47) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.81) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.67) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.15) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.63) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.71) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.59) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.89) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.36) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.21) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.91) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.99) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.87) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.49) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.13) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.15) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.72) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.28) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.22) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.22) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.72) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.78) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.31) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.52) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.70) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.61) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.12) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.55) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.53) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.62) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.41) Agent not enforced to meet deadline. /------------------- | Step 34 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.01) Agent not enforced to meet deadline. /------------------- | Step 35 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.55) Agent not enforced to meet deadline. /------------------- | Step 36 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.64) Agent not enforced to meet deadline. /------------------- | Step 37 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.16) Agent not enforced to meet deadline. /------------------- | Step 38 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.20) Agent not enforced to meet deadline. /------------------- | Step 39 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.66) Agent not enforced to meet deadline. /------------------- | Step 40 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.72) Agent not enforced to meet deadline. /------------------- | Step 41 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.35) Agent not enforced to meet deadline. /------------------- | Step 42 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.20) Agent not enforced to meet deadline. /------------------- | Step 43 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.13) Agent not enforced to meet deadline. /------------------- | Step 44 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.74) Agent not enforced to meet deadline. /------------------- | Step 45 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.85) Agent not enforced to meet deadline. /------------------- | Step 46 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.92) Agent not enforced to meet deadline. /------------------- | Step 47 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.91) Agent not enforced to meet deadline. /------------------- | Step 48 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.72) Agent not enforced to meet deadline. /------------------- | Step 49 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.42) Agent not enforced to meet deadline. /------------------- | Step 50 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.85) Agent not enforced to meet deadline. /------------------- | Step 51 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.98) Agent not enforced to meet deadline. /------------------- | Step 52 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.04) Agent not enforced to meet deadline. /------------------- | Step 53 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.26) Agent not enforced to meet deadline. /------------------- | Step 54 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.21) Agent not enforced to meet deadline. /------------------- | Step 55 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.01) Agent not enforced to meet deadline. /------------------- | Step 56 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.29) Agent not enforced to meet deadline. /------------------- | Step 57 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.82) Agent not enforced to meet deadline. /------------------- | Step 58 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.30) Agent not enforced to meet deadline. /------------------- | Step 59 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.47) Agent not enforced to meet deadline. /------------------- | Step 60 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.56) Agent not enforced to meet deadline. /------------------- | Step 61 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.74) Agent not enforced to meet deadline. /------------------- | Step 62 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.84) Agent not enforced to meet deadline. /------------------- | Step 63 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.62) Agent not enforced to meet deadline. /------------------- | Step 64 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.27) Agent not enforced to meet deadline. /------------------- | Step 65 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.24) Agent not enforced to meet deadline. /------------------- | Step 66 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.45) Agent not enforced to meet deadline. /------------------- | Step 67 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.46) Agent not enforced to meet deadline. /------------------- | Step 68 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.11) Agent not enforced to meet deadline. /------------------- | Step 69 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.94) Agent not enforced to meet deadline. /------------------- | Step 70 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.83) Agent not enforced to meet deadline. /------------------- | Step 71 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.48) Agent not enforced to meet deadline. /------------------- | Step 72 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.93) Agent not enforced to meet deadline. /------------------- | Step 73 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.24) Agent not enforced to meet deadline. /------------------- | Step 74 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.52) Agent not enforced to meet deadline. /------------------- | Step 75 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.50) Agent not enforced to meet deadline. /------------------- | Step 76 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.53) Agent not enforced to meet deadline. /------------------- | Step 77 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.48) Agent not enforced to meet deadline. /------------------- | Step 78 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.71) Agent not enforced to meet deadline. /------------------- | Step 79 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.62) Agent not enforced to meet deadline. /------------------- | Step 80 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.86) Agent not enforced to meet deadline. /------------------- | Step 81 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.93) Agent not enforced to meet deadline. /------------------- | Step 82 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.30) Agent not enforced to meet deadline. /------------------- | Step 83 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.19) Agent not enforced to meet deadline. /------------------- | Step 84 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.33) Agent not enforced to meet deadline. /------------------- | Step 85 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.84) Agent not enforced to meet deadline. /------------------- | Step 86 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.18) Agent not enforced to meet deadline. /------------------- | Step 87 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.71) Agent not enforced to meet deadline. /------------------- | Step 88 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.90) Agent not enforced to meet deadline. /------------------- | Step 89 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.55) Agent not enforced to meet deadline. /------------------- | Step 90 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.43) Agent not enforced to meet deadline. /------------------- | Step 91 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.59) Agent not enforced to meet deadline. /------------------- | Step 92 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.55) Agent not enforced to meet deadline. /------------------- | Step 93 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.76) Agent not enforced to meet deadline. /------------------- | Step 94 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.28) Agent not enforced to meet deadline. /------------------- | Step 95 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.33) Agent not enforced to meet deadline. /------------------- | Step 96 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.86) Agent not enforced to meet deadline. /------------------- | Step 97 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.54) Agent not enforced to meet deadline. /------------------- | Step 98 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.27) Agent not enforced to meet deadline. /------------------- | Step 99 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.87) Agent not enforced to meet deadline. /------------------- | Step 100 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.73) Agent not enforced to meet deadline. /------------------- | Step 101 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.60) Agent not enforced to meet deadline. /------------------- | Step 102 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.71) Agent not enforced to meet deadline. /------------------- | Step 103 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.53) Agent not enforced to meet deadline. /------------------- | Step 104 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.58) Agent not enforced to meet deadline. /------------------- | Step 105 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.02) Agent not enforced to meet deadline. /------------------- | Step 106 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.94) Agent not enforced to meet deadline. /------------------- | Step 107 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.31) Agent not enforced to meet deadline. /------------------- | Step 108 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.35) Agent not enforced to meet deadline. /------------------- | Step 109 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.85) Agent not enforced to meet deadline. /------------------- | Step 110 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.94) Agent not enforced to meet deadline. /------------------- | Step 111 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.14) Agent not enforced to meet deadline. /------------------- | Step 112 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.01) Agent not enforced to meet deadline. /------------------- | Step 113 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.99) Agent not enforced to meet deadline. /------------------- | Step 114 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.84) Agent not enforced to meet deadline. /------------------- | Step 115 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.02) Agent not enforced to meet deadline. /------------------- | Step 116 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.29) Agent not enforced to meet deadline. /------------------- | Step 117 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.75) Agent not enforced to meet deadline. /------------------- | Step 118 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.94) Agent not enforced to meet deadline. /------------------- | Step 119 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.97) Agent not enforced to meet deadline. Trial Aborted! Agent did not reach the destination. /------------------------- | Training trial 4 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.84) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.57) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.76) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.88) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.17) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.87) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.13) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.80) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.43) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.07) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.77) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.33) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.59) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.71) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.11) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Training trial 5 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.28) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.11) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.93) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.64) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.09) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.02) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.48) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.52) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.21) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.16) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.17) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.16) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.89) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.50) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.23) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.80) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.48) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.16) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.09) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.68) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.90) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.34) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.79) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.63) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.18) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -41.00) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.94) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.31) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.05) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.02) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.24) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.15) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.44) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.30) Agent not enforced to meet deadline. /------------------- | Step 34 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.44) Agent not enforced to meet deadline. /------------------- | Step 35 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.74) Agent not enforced to meet deadline. /------------------- | Step 36 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.51) Agent not enforced to meet deadline. /------------------- | Step 37 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.23) Agent not enforced to meet deadline. /------------------- | Step 38 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.70) Agent not enforced to meet deadline. /------------------- | Step 39 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.04) Agent not enforced to meet deadline. /------------------- | Step 40 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.67) Agent not enforced to meet deadline. /------------------- | Step 41 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.36) Agent not enforced to meet deadline. /------------------- | Step 42 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.48) Agent not enforced to meet deadline. /------------------- | Step 43 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.11) Agent not enforced to meet deadline. /------------------- | Step 44 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.02) Agent not enforced to meet deadline. /------------------- | Step 45 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.93) Agent not enforced to meet deadline. /------------------- | Step 46 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.09) Agent not enforced to meet deadline. /------------------- | Step 47 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.44) Agent not enforced to meet deadline. /------------------- | Step 48 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.02) Agent not enforced to meet deadline. /------------------- | Step 49 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.89) Agent not enforced to meet deadline. /------------------- | Step 50 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.13) Agent not enforced to meet deadline. /------------------- | Step 51 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.15) Agent not enforced to meet deadline. /------------------- | Step 52 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.72) Agent not enforced to meet deadline. /------------------- | Step 53 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.81) Agent not enforced to meet deadline. /------------------- | Step 54 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.01) Agent not enforced to meet deadline. /------------------- | Step 55 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.33) Agent not enforced to meet deadline. /------------------- | Step 56 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.11) Agent not enforced to meet deadline. /------------------- | Step 57 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.68) Agent not enforced to meet deadline. /------------------- | Step 58 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.87) Agent not enforced to meet deadline. /------------------- | Step 59 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.60) Agent not enforced to meet deadline. /------------------- | Step 60 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.85) Agent not enforced to meet deadline. /------------------- | Step 61 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.18) Agent not enforced to meet deadline. /------------------- | Step 62 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.94) Agent not enforced to meet deadline. /------------------- | Step 63 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.20) Agent not enforced to meet deadline. /------------------- | Step 64 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.39) Agent not enforced to meet deadline. /------------------- | Step 65 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.07) Agent not enforced to meet deadline. /------------------- | Step 66 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.44) Agent not enforced to meet deadline. /------------------- | Step 67 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.91) Agent not enforced to meet deadline. /------------------- | Step 68 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.48) Agent not enforced to meet deadline. /------------------- | Step 69 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.17) Agent not enforced to meet deadline. /------------------- | Step 70 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.92) Agent not enforced to meet deadline. /------------------- | Step 71 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.20) Agent not enforced to meet deadline. /------------------- | Step 72 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.34) Agent not enforced to meet deadline. /------------------- | Step 73 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.54) Agent not enforced to meet deadline. /------------------- | Step 74 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.73) Agent not enforced to meet deadline. /------------------- | Step 75 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.71) Agent not enforced to meet deadline. /------------------- | Step 76 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.46) Agent not enforced to meet deadline. /------------------- | Step 77 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.90) Agent not enforced to meet deadline. /------------------- | Step 78 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.61) Agent not enforced to meet deadline. /------------------- | Step 79 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.19) Agent not enforced to meet deadline. /------------------- | Step 80 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.19) Agent not enforced to meet deadline. /------------------- | Step 81 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.22) Agent not enforced to meet deadline. /------------------- | Step 82 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.06) Agent not enforced to meet deadline. /------------------- | Step 83 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.11) Agent not enforced to meet deadline. /------------------- | Step 84 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.21) Agent not enforced to meet deadline. /------------------- | Step 85 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.53) Agent not enforced to meet deadline. /------------------- | Step 86 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.20) Agent not enforced to meet deadline. /------------------- | Step 87 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.05) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Training trial 6 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.25) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.31) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.80) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.46) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.78) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.36) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.47) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.40) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.11) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.53) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.37) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.03) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.48) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Training trial 7 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.80) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.45) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.50) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.02) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.14) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.90) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.59) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.64) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.79) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.67) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.00) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.58) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.99) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.89) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.69) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.40) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.60) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.00) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.57) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.73) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.02) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.35) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.26) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.91) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.89) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.61) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.48) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.47) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.63) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.99) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.32) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.04) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.06) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.67) Agent not enforced to meet deadline. /------------------- | Step 34 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.44) Agent not enforced to meet deadline. /------------------- | Step 35 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.39) Agent not enforced to meet deadline. /------------------- | Step 36 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.85) Agent not enforced to meet deadline. /------------------- | Step 37 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.09) Agent not enforced to meet deadline. /------------------- | Step 38 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.09) Agent not enforced to meet deadline. /------------------- | Step 39 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.24) Agent not enforced to meet deadline. /------------------- | Step 40 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.87) Agent not enforced to meet deadline. /------------------- | Step 41 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.63) Agent not enforced to meet deadline. /------------------- | Step 42 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.11) Agent not enforced to meet deadline. /------------------- | Step 43 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.11) Agent not enforced to meet deadline. /------------------- | Step 44 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.24) Agent not enforced to meet deadline. /------------------- | Step 45 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.50) Agent not enforced to meet deadline. /------------------- | Step 46 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.26) Agent not enforced to meet deadline. /------------------- | Step 47 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.38) Agent not enforced to meet deadline. /------------------- | Step 48 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.36) Agent not enforced to meet deadline. /------------------- | Step 49 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.92) Agent not enforced to meet deadline. /------------------- | Step 50 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.78) Agent not enforced to meet deadline. /------------------- | Step 51 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.66) Agent not enforced to meet deadline. /------------------- | Step 52 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.91) Agent not enforced to meet deadline. /------------------- | Step 53 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.73) Agent not enforced to meet deadline. /------------------- | Step 54 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.49) Agent not enforced to meet deadline. /------------------- | Step 55 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.41) Agent not enforced to meet deadline. /------------------- | Step 56 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.77) Agent not enforced to meet deadline. /------------------- | Step 57 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.65) Agent not enforced to meet deadline. /------------------- | Step 58 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.49) Agent not enforced to meet deadline. /------------------- | Step 59 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.23) Agent not enforced to meet deadline. /------------------- | Step 60 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.08) Agent not enforced to meet deadline. /------------------- | Step 61 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.54) Agent not enforced to meet deadline. /------------------- | Step 62 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.44) Agent not enforced to meet deadline. /------------------- | Step 63 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.74) Agent not enforced to meet deadline. /------------------- | Step 64 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.97) Agent not enforced to meet deadline. /------------------- | Step 65 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.26) Agent not enforced to meet deadline. /------------------- | Step 66 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.95) Agent not enforced to meet deadline. /------------------- | Step 67 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.09) Agent not enforced to meet deadline. /------------------- | Step 68 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.13) Agent not enforced to meet deadline. /------------------- | Step 69 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.07) Agent not enforced to meet deadline. /------------------- | Step 70 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.73) Agent not enforced to meet deadline. /------------------- | Step 71 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.69) Agent not enforced to meet deadline. /------------------- | Step 72 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.34) Agent not enforced to meet deadline. /------------------- | Step 73 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.86) Agent not enforced to meet deadline. /------------------- | Step 74 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.26) Agent not enforced to meet deadline. /------------------- | Step 75 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.48) Agent not enforced to meet deadline. /------------------- | Step 76 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.04) Agent not enforced to meet deadline. /------------------- | Step 77 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.67) Agent not enforced to meet deadline. /------------------- | Step 78 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.97) Agent not enforced to meet deadline. /------------------- | Step 79 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.84) Agent not enforced to meet deadline. /------------------- | Step 80 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.56) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Training trial 8 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.52) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.84) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.04) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.87) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.78) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.36) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.21) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.96) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.58) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.30) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.33) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.04) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.51) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.38) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.85) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.60) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.21) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.25) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.76) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.21) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.90) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.66) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.51) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.80) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.15) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.25) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.04) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.54) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.10) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.20) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.87) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.25) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.71) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.78) Agent not enforced to meet deadline. /------------------- | Step 34 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.66) Agent not enforced to meet deadline. /------------------- | Step 35 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.32) Agent not enforced to meet deadline. /------------------- | Step 36 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.68) Agent not enforced to meet deadline. /------------------- | Step 37 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.39) Agent not enforced to meet deadline. /------------------- | Step 38 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.97) Agent not enforced to meet deadline. /------------------- | Step 39 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.60) Agent not enforced to meet deadline. /------------------- | Step 40 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.31) Agent not enforced to meet deadline. /------------------- | Step 41 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.85) Agent not enforced to meet deadline. /------------------- | Step 42 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.81) Agent not enforced to meet deadline. /------------------- | Step 43 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.64) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Training trial 9 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.20) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.95) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.85) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.95) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.04) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.07) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.29) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.90) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.94) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.10) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.22) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.32) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.45) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.02) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.58) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.42) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.02) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.80) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.14) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.60) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.83) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.85) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.30) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.42) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.85) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.45) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.19) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.65) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.14) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.58) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.70) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.58) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.99) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.02) Agent not enforced to meet deadline. /------------------- | Step 34 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.84) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Training trial 10 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.72) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.41) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.44) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.78) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.58) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.96) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.37) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.02) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.69) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.58) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.02) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.17) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.01) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.90) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.36) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.14) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.76) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.50) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.34) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.56) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.21) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Training trial 11 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.80) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.16) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.58) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.55) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.93) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.47) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.50) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.32) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.73) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.98) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.50) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.17) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.17) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.12) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.23) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.99) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.98) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.79) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.64) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.75) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.09) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.02) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.23) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.01) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.19) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.55) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.19) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.27) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.05) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.47) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.98) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.21) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.11) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.09) Agent not enforced to meet deadline. /------------------- | Step 34 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.14) Agent not enforced to meet deadline. /------------------- | Step 35 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.72) Agent not enforced to meet deadline. /------------------- | Step 36 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.49) Agent not enforced to meet deadline. /------------------- | Step 37 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.70) Agent not enforced to meet deadline. /------------------- | Step 38 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.39) Agent not enforced to meet deadline. /------------------- | Step 39 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.22) Agent not enforced to meet deadline. /------------------- | Step 40 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.43) Agent not enforced to meet deadline. /------------------- | Step 41 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.20) Agent not enforced to meet deadline. /------------------- | Step 42 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.55) Agent not enforced to meet deadline. /------------------- | Step 43 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.17) Agent not enforced to meet deadline. /------------------- | Step 44 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 3.00) Agent not enforced to meet deadline. /------------------- | Step 45 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.71) Agent not enforced to meet deadline. /------------------- | Step 46 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.31) Agent not enforced to meet deadline. /------------------- | Step 47 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.33) Agent not enforced to meet deadline. /------------------- | Step 48 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.05) Agent not enforced to meet deadline. /------------------- | Step 49 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.15) Agent not enforced to meet deadline. /------------------- | Step 50 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.38) Agent not enforced to meet deadline. /------------------- | Step 51 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.23) Agent not enforced to meet deadline. /------------------- | Step 52 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.98) Agent not enforced to meet deadline. /------------------- | Step 53 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.70) Agent not enforced to meet deadline. /------------------- | Step 54 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.11) Agent not enforced to meet deadline. /------------------- | Step 55 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.12) Agent not enforced to meet deadline. /------------------- | Step 56 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.66) Agent not enforced to meet deadline. /------------------- | Step 57 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.33) Agent not enforced to meet deadline. /------------------- | Step 58 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.91) Agent not enforced to meet deadline. /------------------- | Step 59 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.95) Agent not enforced to meet deadline. /------------------- | Step 60 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.72) Agent not enforced to meet deadline. /------------------- | Step 61 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.26) Agent not enforced to meet deadline. /------------------- | Step 62 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.49) Agent not enforced to meet deadline. /------------------- | Step 63 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.55) Agent not enforced to meet deadline. /------------------- | Step 64 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.87) Agent not enforced to meet deadline. /------------------- | Step 65 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.02) Agent not enforced to meet deadline. /------------------- | Step 66 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.41) Agent not enforced to meet deadline. /------------------- | Step 67 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.49) Agent not enforced to meet deadline. /------------------- | Step 68 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.78) Agent not enforced to meet deadline. /------------------- | Step 69 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.76) Agent not enforced to meet deadline. /------------------- | Step 70 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.59) Agent not enforced to meet deadline. /------------------- | Step 71 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.38) Agent not enforced to meet deadline. /------------------- | Step 72 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.30) Agent not enforced to meet deadline. /------------------- | Step 73 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.10) Agent not enforced to meet deadline. /------------------- | Step 74 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.96) Agent not enforced to meet deadline. /------------------- | Step 75 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.84) Agent not enforced to meet deadline. /------------------- | Step 76 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.42) Agent not enforced to meet deadline. /------------------- | Step 77 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.29) Agent not enforced to meet deadline. /------------------- | Step 78 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.02) Agent not enforced to meet deadline. /------------------- | Step 79 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.67) Agent not enforced to meet deadline. /------------------- | Step 80 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.99) Agent not enforced to meet deadline. /------------------- | Step 81 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.67) Agent not enforced to meet deadline. /------------------- | Step 82 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.78) Agent not enforced to meet deadline. /------------------- | Step 83 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.00) Agent not enforced to meet deadline. /------------------- | Step 84 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.26) Agent not enforced to meet deadline. /------------------- | Step 85 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.02) Agent not enforced to meet deadline. /------------------- | Step 86 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.96) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Training trial 12 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.73) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.89) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.22) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.73) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.22) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.03) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.57) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.04) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.18) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.90) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.85) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.69) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.76) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.85) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.37) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.20) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.24) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.55) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.75) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.73) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.16) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.73) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.79) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.52) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.38) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.68) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.38) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.42) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.31) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.18) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.26) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.88) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.75) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.36) Agent not enforced to meet deadline. /------------------- | Step 34 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.21) Agent not enforced to meet deadline. /------------------- | Step 35 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.69) Agent not enforced to meet deadline. /------------------- | Step 36 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.01) Agent not enforced to meet deadline. /------------------- | Step 37 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.69) Agent not enforced to meet deadline. /------------------- | Step 38 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.72) Agent not enforced to meet deadline. /------------------- | Step 39 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.83) Agent not enforced to meet deadline. /------------------- | Step 40 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.35) Agent not enforced to meet deadline. /------------------- | Step 41 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.24) Agent not enforced to meet deadline. /------------------- | Step 42 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.27) Agent not enforced to meet deadline. /------------------- | Step 43 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.01) Agent not enforced to meet deadline. /------------------- | Step 44 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.42) Agent not enforced to meet deadline. /------------------- | Step 45 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.38) Agent not enforced to meet deadline. /------------------- | Step 46 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.79) Agent not enforced to meet deadline. /------------------- | Step 47 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.04) Agent not enforced to meet deadline. /------------------- | Step 48 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.63) Agent not enforced to meet deadline. /------------------- | Step 49 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.27) Agent not enforced to meet deadline. /------------------- | Step 50 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.48) Agent not enforced to meet deadline. /------------------- | Step 51 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.41) Agent not enforced to meet deadline. /------------------- | Step 52 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.73) Agent not enforced to meet deadline. /------------------- | Step 53 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.40) Agent not enforced to meet deadline. /------------------- | Step 54 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.65) Agent not enforced to meet deadline. /------------------- | Step 55 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.08) Agent not enforced to meet deadline. /------------------- | Step 56 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.44) Agent not enforced to meet deadline. /------------------- | Step 57 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.85) Agent not enforced to meet deadline. /------------------- | Step 58 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.04) Agent not enforced to meet deadline. /------------------- | Step 59 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.09) Agent not enforced to meet deadline. /------------------- | Step 60 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.50) Agent not enforced to meet deadline. /------------------- | Step 61 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.07) Agent not enforced to meet deadline. /------------------- | Step 62 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.35) Agent not enforced to meet deadline. /------------------- | Step 63 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.93) Agent not enforced to meet deadline. /------------------- | Step 64 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.33) Agent not enforced to meet deadline. /------------------- | Step 65 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.21) Agent not enforced to meet deadline. /------------------- | Step 66 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.06) Agent not enforced to meet deadline. /------------------- | Step 67 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.01) Agent not enforced to meet deadline. /------------------- | Step 68 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.27) Agent not enforced to meet deadline. /------------------- | Step 69 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.95) Agent not enforced to meet deadline. /------------------- | Step 70 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.76) Agent not enforced to meet deadline. /------------------- | Step 71 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.50) Agent not enforced to meet deadline. /------------------- | Step 72 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.27) Agent not enforced to meet deadline. /------------------- | Step 73 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.01) Agent not enforced to meet deadline. /------------------- | Step 74 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.89) Agent not enforced to meet deadline. /------------------- | Step 75 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.18) Agent not enforced to meet deadline. /------------------- | Step 76 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.66) Agent not enforced to meet deadline. /------------------- | Step 77 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.58) Agent not enforced to meet deadline. /------------------- | Step 78 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.34) Agent not enforced to meet deadline. /------------------- | Step 79 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.04) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Training trial 13 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.13) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.39) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.34) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.24) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.50) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.67) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.31) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.65) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.49) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.68) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.59) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.62) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.20) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.57) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.58) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.46) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Training trial 14 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.59) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.64) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.39) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.02) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.15) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.28) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.88) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.02) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.66) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.22) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.27) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.42) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.16) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.39) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.64) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.43) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.58) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.19) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.88) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.33) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.73) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.88) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.70) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.63) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.77) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.38) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.21) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.63) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.93) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.41) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.97) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.93) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.40) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.20) Agent not enforced to meet deadline. /------------------- | Step 34 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.96) Agent not enforced to meet deadline. /------------------- | Step 35 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.44) Agent not enforced to meet deadline. /------------------- | Step 36 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.98) Agent not enforced to meet deadline. /------------------- | Step 37 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.85) Agent not enforced to meet deadline. /------------------- | Step 38 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.97) Agent not enforced to meet deadline. /------------------- | Step 39 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.57) Agent not enforced to meet deadline. /------------------- | Step 40 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.19) Agent not enforced to meet deadline. /------------------- | Step 41 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.82) Agent not enforced to meet deadline. /------------------- | Step 42 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.34) Agent not enforced to meet deadline. /------------------- | Step 43 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.85) Agent not enforced to meet deadline. /------------------- | Step 44 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.84) Agent not enforced to meet deadline. /------------------- | Step 45 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.26) Agent not enforced to meet deadline. /------------------- | Step 46 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.77) Agent not enforced to meet deadline. /------------------- | Step 47 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.39) Agent not enforced to meet deadline. /------------------- | Step 48 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.32) Agent not enforced to meet deadline. /------------------- | Step 49 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.38) Agent not enforced to meet deadline. /------------------- | Step 50 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.40) Agent not enforced to meet deadline. /------------------- | Step 51 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.06) Agent not enforced to meet deadline. /------------------- | Step 52 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.49) Agent not enforced to meet deadline. /------------------- | Step 53 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.60) Agent not enforced to meet deadline. /------------------- | Step 54 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.69) Agent not enforced to meet deadline. /------------------- | Step 55 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.14) Agent not enforced to meet deadline. /------------------- | Step 56 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.54) Agent not enforced to meet deadline. /------------------- | Step 57 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.95) Agent not enforced to meet deadline. /------------------- | Step 58 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.57) Agent not enforced to meet deadline. /------------------- | Step 59 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.31) Agent not enforced to meet deadline. /------------------- | Step 60 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.83) Agent not enforced to meet deadline. /------------------- | Step 61 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.00) Agent not enforced to meet deadline. /------------------- | Step 62 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.54) Agent not enforced to meet deadline. /------------------- | Step 63 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.38) Agent not enforced to meet deadline. /------------------- | Step 64 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.74) Agent not enforced to meet deadline. /------------------- | Step 65 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.34) Agent not enforced to meet deadline. /------------------- | Step 66 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.07) Agent not enforced to meet deadline. /------------------- | Step 67 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.92) Agent not enforced to meet deadline. /------------------- | Step 68 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.92) Agent not enforced to meet deadline. /------------------- | Step 69 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.51) Agent not enforced to meet deadline. /------------------- | Step 70 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.06) Agent not enforced to meet deadline. /------------------- | Step 71 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.10) Agent not enforced to meet deadline. /------------------- | Step 72 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.35) Agent not enforced to meet deadline. /------------------- | Step 73 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.41) Agent not enforced to meet deadline. /------------------- | Step 74 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.37) Agent not enforced to meet deadline. /------------------- | Step 75 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.09) Agent not enforced to meet deadline. /------------------- | Step 76 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.16) Agent not enforced to meet deadline. /------------------- | Step 77 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.04) Agent not enforced to meet deadline. /------------------- | Step 78 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.86) Agent not enforced to meet deadline. /------------------- | Step 79 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.79) Agent not enforced to meet deadline. /------------------- | Step 80 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.05) Agent not enforced to meet deadline. /------------------- | Step 81 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.91) Agent not enforced to meet deadline. /------------------- | Step 82 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.41) Agent not enforced to meet deadline. /------------------- | Step 83 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.46) Agent not enforced to meet deadline. /------------------- | Step 84 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.42) Agent not enforced to meet deadline. /------------------- | Step 85 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.84) Agent not enforced to meet deadline. /------------------- | Step 86 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.73) Agent not enforced to meet deadline. /------------------- | Step 87 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.09) Agent not enforced to meet deadline. /------------------- | Step 88 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.79) Agent not enforced to meet deadline. /------------------- | Step 89 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.79) Agent not enforced to meet deadline. /------------------- | Step 90 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.57) Agent not enforced to meet deadline. /------------------- | Step 91 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.20) Agent not enforced to meet deadline. /------------------- | Step 92 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.87) Agent not enforced to meet deadline. /------------------- | Step 93 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.85) Agent not enforced to meet deadline. /------------------- | Step 94 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.93) Agent not enforced to meet deadline. /------------------- | Step 95 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.90) Agent not enforced to meet deadline. /------------------- | Step 96 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.30) Agent not enforced to meet deadline. /------------------- | Step 97 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.27) Agent not enforced to meet deadline. /------------------- | Step 98 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.46) Agent not enforced to meet deadline. /------------------- | Step 99 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.76) Agent not enforced to meet deadline. /------------------- | Step 100 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.92) Agent not enforced to meet deadline. /------------------- | Step 101 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.18) Agent not enforced to meet deadline. /------------------- | Step 102 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.50) Agent not enforced to meet deadline. /------------------- | Step 103 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.25) Agent not enforced to meet deadline. /------------------- | Step 104 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.69) Agent not enforced to meet deadline. /------------------- | Step 105 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.45) Agent not enforced to meet deadline. /------------------- | Step 106 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.42) Agent not enforced to meet deadline. /------------------- | Step 107 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.17) Agent not enforced to meet deadline. /------------------- | Step 108 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.57) Agent not enforced to meet deadline. /------------------- | Step 109 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.39) Agent not enforced to meet deadline. /------------------- | Step 110 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.65) Agent not enforced to meet deadline. /------------------- | Step 111 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.75) Agent not enforced to meet deadline. /------------------- | Step 112 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.21) Agent not enforced to meet deadline. /------------------- | Step 113 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.72) Agent not enforced to meet deadline. /------------------- | Step 114 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.81) Agent not enforced to meet deadline. /------------------- | Step 115 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.29) Agent not enforced to meet deadline. /------------------- | Step 116 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.53) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Training trial 15 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.69) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.40) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.33) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.00) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.78) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.35) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.42) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.67) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.93) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.20) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.73) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.94) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.18) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.51) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.38) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.49) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.44) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.35) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.88) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.11) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.38) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.35) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.97) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.70) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.82) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.51) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.71) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.67) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.65) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.06) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.91) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.09) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.70) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.70) Agent not enforced to meet deadline. /------------------- | Step 34 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.12) Agent not enforced to meet deadline. /------------------- | Step 35 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.37) Agent not enforced to meet deadline. /------------------- | Step 36 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.08) Agent not enforced to meet deadline. /------------------- | Step 37 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.49) Agent not enforced to meet deadline. /------------------- | Step 38 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.77) Agent not enforced to meet deadline. /------------------- | Step 39 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.17) Agent not enforced to meet deadline. /------------------- | Step 40 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.27) Agent not enforced to meet deadline. /------------------- | Step 41 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.83) Agent not enforced to meet deadline. /------------------- | Step 42 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.54) Agent not enforced to meet deadline. /------------------- | Step 43 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.52) Agent not enforced to meet deadline. /------------------- | Step 44 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.62) Agent not enforced to meet deadline. /------------------- | Step 45 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.99) Agent not enforced to meet deadline. /------------------- | Step 46 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.18) Agent not enforced to meet deadline. /------------------- | Step 47 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.15) Agent not enforced to meet deadline. /------------------- | Step 48 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.60) Agent not enforced to meet deadline. /------------------- | Step 49 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.52) Agent not enforced to meet deadline. /------------------- | Step 50 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.74) Agent not enforced to meet deadline. /------------------- | Step 51 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.99) Agent not enforced to meet deadline. /------------------- | Step 52 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.42) Agent not enforced to meet deadline. /------------------- | Step 53 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.07) Agent not enforced to meet deadline. /------------------- | Step 54 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.63) Agent not enforced to meet deadline. /------------------- | Step 55 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.71) Agent not enforced to meet deadline. /------------------- | Step 56 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.91) Agent not enforced to meet deadline. /------------------- | Step 57 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.30) Agent not enforced to meet deadline. /------------------- | Step 58 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.71) Agent not enforced to meet deadline. /------------------- | Step 59 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.54) Agent not enforced to meet deadline. /------------------- | Step 60 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.58) Agent not enforced to meet deadline. /------------------- | Step 61 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.14) Agent not enforced to meet deadline. /------------------- | Step 62 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.77) Agent not enforced to meet deadline. /------------------- | Step 63 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.32) Agent not enforced to meet deadline. /------------------- | Step 64 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.14) Agent not enforced to meet deadline. /------------------- | Step 65 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.19) Agent not enforced to meet deadline. /------------------- | Step 66 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.19) Agent not enforced to meet deadline. /------------------- | Step 67 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.62) Agent not enforced to meet deadline. /------------------- | Step 68 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.62) Agent not enforced to meet deadline. /------------------- | Step 69 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.74) Agent not enforced to meet deadline. /------------------- | Step 70 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.48) Agent not enforced to meet deadline. /------------------- | Step 71 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.65) Agent not enforced to meet deadline. /------------------- | Step 72 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.59) Agent not enforced to meet deadline. /------------------- | Step 73 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.27) Agent not enforced to meet deadline. /------------------- | Step 74 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.83) Agent not enforced to meet deadline. /------------------- | Step 75 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.01) Agent not enforced to meet deadline. /------------------- | Step 76 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.37) Agent not enforced to meet deadline. /------------------- | Step 77 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.44) Agent not enforced to meet deadline. /------------------- | Step 78 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.54) Agent not enforced to meet deadline. /------------------- | Step 79 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.76) Agent not enforced to meet deadline. /------------------- | Step 80 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.47) Agent not enforced to meet deadline. /------------------- | Step 81 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.98) Agent not enforced to meet deadline. /------------------- | Step 82 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.39) Agent not enforced to meet deadline. /------------------- | Step 83 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.87) Agent not enforced to meet deadline. /------------------- | Step 84 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.35) Agent not enforced to meet deadline. /------------------- | Step 85 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.36) Agent not enforced to meet deadline. /------------------- | Step 86 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.12) Agent not enforced to meet deadline. /------------------- | Step 87 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.74) Agent not enforced to meet deadline. /------------------- | Step 88 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.52) Agent not enforced to meet deadline. /------------------- | Step 89 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.03) Agent not enforced to meet deadline. /------------------- | Step 90 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.09) Agent not enforced to meet deadline. /------------------- | Step 91 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.44) Agent not enforced to meet deadline. /------------------- | Step 92 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.72) Agent not enforced to meet deadline. /------------------- | Step 93 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.38) Agent not enforced to meet deadline. /------------------- | Step 94 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.77) Agent not enforced to meet deadline. /------------------- | Step 95 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.64) Agent not enforced to meet deadline. /------------------- | Step 96 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.32) Agent not enforced to meet deadline. /------------------- | Step 97 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.47) Agent not enforced to meet deadline. /------------------- | Step 98 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.54) Agent not enforced to meet deadline. /------------------- | Step 99 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.78) Agent not enforced to meet deadline. /------------------- | Step 100 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.70) Agent not enforced to meet deadline. /------------------- | Step 101 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.73) Agent not enforced to meet deadline. /------------------- | Step 102 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.90) Agent not enforced to meet deadline. /------------------- | Step 103 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.65) Agent not enforced to meet deadline. /------------------- | Step 104 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.27) Agent not enforced to meet deadline. /------------------- | Step 105 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.28) Agent not enforced to meet deadline. /------------------- | Step 106 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.35) Agent not enforced to meet deadline. /------------------- | Step 107 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.12) Agent not enforced to meet deadline. /------------------- | Step 108 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.71) Agent not enforced to meet deadline. /------------------- | Step 109 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.35) Agent not enforced to meet deadline. /------------------- | Step 110 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.57) Agent not enforced to meet deadline. /------------------- | Step 111 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.16) Agent not enforced to meet deadline. /------------------- | Step 112 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.47) Agent not enforced to meet deadline. /------------------- | Step 113 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.13) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Training trial 16 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.17) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.19) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.58) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.28) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.16) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.96) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.53) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.06) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.81) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.17) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.46) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.37) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.93) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.19) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.84) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.51) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.82) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.00) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.63) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.00) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.55) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.40) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.48) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.68) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.21) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.38) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.32) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.55) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.92) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.01) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.34) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.02) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.94) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.86) Agent not enforced to meet deadline. /------------------- | Step 34 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.30) Agent not enforced to meet deadline. /------------------- | Step 35 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.02) Agent not enforced to meet deadline. /------------------- | Step 36 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.09) Agent not enforced to meet deadline. /------------------- | Step 37 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.23) Agent not enforced to meet deadline. /------------------- | Step 38 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.43) Agent not enforced to meet deadline. /------------------- | Step 39 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.91) Agent not enforced to meet deadline. /------------------- | Step 40 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.77) Agent not enforced to meet deadline. /------------------- | Step 41 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.20) Agent not enforced to meet deadline. /------------------- | Step 42 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.09) Agent not enforced to meet deadline. /------------------- | Step 43 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.57) Agent not enforced to meet deadline. /------------------- | Step 44 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.50) Agent not enforced to meet deadline. /------------------- | Step 45 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.32) Agent not enforced to meet deadline. /------------------- | Step 46 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.46) Agent not enforced to meet deadline. /------------------- | Step 47 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.85) Agent not enforced to meet deadline. /------------------- | Step 48 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.83) Agent not enforced to meet deadline. /------------------- | Step 49 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.67) Agent not enforced to meet deadline. /------------------- | Step 50 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.01) Agent not enforced to meet deadline. /------------------- | Step 51 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.79) Agent not enforced to meet deadline. /------------------- | Step 52 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.71) Agent not enforced to meet deadline. /------------------- | Step 53 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.41) Agent not enforced to meet deadline. /------------------- | Step 54 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.40) Agent not enforced to meet deadline. /------------------- | Step 55 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.25) Agent not enforced to meet deadline. /------------------- | Step 56 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.50) Agent not enforced to meet deadline. /------------------- | Step 57 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.81) Agent not enforced to meet deadline. /------------------- | Step 58 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.40) Agent not enforced to meet deadline. /------------------- | Step 59 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.92) Agent not enforced to meet deadline. /------------------- | Step 60 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.24) Agent not enforced to meet deadline. /------------------- | Step 61 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.85) Agent not enforced to meet deadline. /------------------- | Step 62 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.08) Agent not enforced to meet deadline. /------------------- | Step 63 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.03) Agent not enforced to meet deadline. /------------------- | Step 64 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.68) Agent not enforced to meet deadline. /------------------- | Step 65 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.30) Agent not enforced to meet deadline. /------------------- | Step 66 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.31) Agent not enforced to meet deadline. /------------------- | Step 67 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.93) Agent not enforced to meet deadline. /------------------- | Step 68 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.86) Agent not enforced to meet deadline. /------------------- | Step 69 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.21) Agent not enforced to meet deadline. /------------------- | Step 70 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.41) Agent not enforced to meet deadline. /------------------- | Step 71 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.06) Agent not enforced to meet deadline. /------------------- | Step 72 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.62) Agent not enforced to meet deadline. /------------------- | Step 73 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.17) Agent not enforced to meet deadline. /------------------- | Step 74 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.88) Agent not enforced to meet deadline. /------------------- | Step 75 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.40) Agent not enforced to meet deadline. /------------------- | Step 76 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.64) Agent not enforced to meet deadline. /------------------- | Step 77 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.11) Agent not enforced to meet deadline. /------------------- | Step 78 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.88) Agent not enforced to meet deadline. /------------------- | Step 79 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.26) Agent not enforced to meet deadline. /------------------- | Step 80 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.01) Agent not enforced to meet deadline. /------------------- | Step 81 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.24) Agent not enforced to meet deadline. /------------------- | Step 82 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.91) Agent not enforced to meet deadline. /------------------- | Step 83 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.85) Agent not enforced to meet deadline. /------------------- | Step 84 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.31) Agent not enforced to meet deadline. /------------------- | Step 85 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.89) Agent not enforced to meet deadline. /------------------- | Step 86 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.41) Agent not enforced to meet deadline. /------------------- | Step 87 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.45) Agent not enforced to meet deadline. /------------------- | Step 88 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.12) Agent not enforced to meet deadline. /------------------- | Step 89 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.41) Agent not enforced to meet deadline. /------------------- | Step 90 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.23) Agent not enforced to meet deadline. /------------------- | Step 91 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.97) Agent not enforced to meet deadline. /------------------- | Step 92 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.79) Agent not enforced to meet deadline. /------------------- | Step 93 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.82) Agent not enforced to meet deadline. /------------------- | Step 94 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.77) Agent not enforced to meet deadline. /------------------- | Step 95 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.65) Agent not enforced to meet deadline. /------------------- | Step 96 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.82) Agent not enforced to meet deadline. /------------------- | Step 97 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.83) Agent not enforced to meet deadline. /------------------- | Step 98 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.84) Agent not enforced to meet deadline. /------------------- | Step 99 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.19) Agent not enforced to meet deadline. /------------------- | Step 100 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.78) Agent not enforced to meet deadline. /------------------- | Step 101 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.83) Agent not enforced to meet deadline. /------------------- | Step 102 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.63) Agent not enforced to meet deadline. /------------------- | Step 103 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.09) Agent not enforced to meet deadline. /------------------- | Step 104 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.34) Agent not enforced to meet deadline. /------------------- | Step 105 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.55) Agent not enforced to meet deadline. /------------------- | Step 106 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.11) Agent not enforced to meet deadline. /------------------- | Step 107 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.09) Agent not enforced to meet deadline. /------------------- | Step 108 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.87) Agent not enforced to meet deadline. /------------------- | Step 109 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.68) Agent not enforced to meet deadline. /------------------- | Step 110 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.30) Agent not enforced to meet deadline. /------------------- | Step 111 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.48) Agent not enforced to meet deadline. /------------------- | Step 112 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.70) Agent not enforced to meet deadline. /------------------- | Step 113 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.12) Agent not enforced to meet deadline. /------------------- | Step 114 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.56) Agent not enforced to meet deadline. /------------------- | Step 115 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.69) Agent not enforced to meet deadline. /------------------- | Step 116 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.39) Agent not enforced to meet deadline. /------------------- | Step 117 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.95) Agent not enforced to meet deadline. /------------------- | Step 118 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.03) Agent not enforced to meet deadline. /------------------- | Step 119 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.08) Agent not enforced to meet deadline. Trial Aborted! Agent did not reach the destination. /------------------------- | Training trial 17 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.97) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.52) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.02) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.23) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.76) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.26) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.80) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -41.00) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.97) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.19) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.85) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.74) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.64) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.53) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.72) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.98) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.27) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.38) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.41) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.29) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.07) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.06) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.58) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.25) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.05) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.11) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.13) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.75) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.31) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.74) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.78) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.38) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.52) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.77) Agent not enforced to meet deadline. /------------------- | Step 34 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.48) Agent not enforced to meet deadline. /------------------- | Step 35 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.72) Agent not enforced to meet deadline. /------------------- | Step 36 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.48) Agent not enforced to meet deadline. /------------------- | Step 37 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.16) Agent not enforced to meet deadline. /------------------- | Step 38 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.66) Agent not enforced to meet deadline. /------------------- | Step 39 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.33) Agent not enforced to meet deadline. /------------------- | Step 40 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.78) Agent not enforced to meet deadline. /------------------- | Step 41 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.78) Agent not enforced to meet deadline. /------------------- | Step 42 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.36) Agent not enforced to meet deadline. /------------------- | Step 43 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.78) Agent not enforced to meet deadline. /------------------- | Step 44 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.66) Agent not enforced to meet deadline. /------------------- | Step 45 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.01) Agent not enforced to meet deadline. /------------------- | Step 46 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.14) Agent not enforced to meet deadline. /------------------- | Step 47 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.32) Agent not enforced to meet deadline. /------------------- | Step 48 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.27) Agent not enforced to meet deadline. /------------------- | Step 49 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.22) Agent not enforced to meet deadline. /------------------- | Step 50 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.91) Agent not enforced to meet deadline. /------------------- | Step 51 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.55) Agent not enforced to meet deadline. /------------------- | Step 52 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.17) Agent not enforced to meet deadline. /------------------- | Step 53 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.48) Agent not enforced to meet deadline. /------------------- | Step 54 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.06) Agent not enforced to meet deadline. /------------------- | Step 55 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.72) Agent not enforced to meet deadline. /------------------- | Step 56 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.95) Agent not enforced to meet deadline. /------------------- | Step 57 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.57) Agent not enforced to meet deadline. /------------------- | Step 58 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.11) Agent not enforced to meet deadline. /------------------- | Step 59 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.97) Agent not enforced to meet deadline. /------------------- | Step 60 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.50) Agent not enforced to meet deadline. /------------------- | Step 61 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.13) Agent not enforced to meet deadline. /------------------- | Step 62 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.67) Agent not enforced to meet deadline. /------------------- | Step 63 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.43) Agent not enforced to meet deadline. /------------------- | Step 64 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.19) Agent not enforced to meet deadline. /------------------- | Step 65 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.10) Agent not enforced to meet deadline. /------------------- | Step 66 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.67) Agent not enforced to meet deadline. /------------------- | Step 67 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.94) Agent not enforced to meet deadline. /------------------- | Step 68 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.21) Agent not enforced to meet deadline. /------------------- | Step 69 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.56) Agent not enforced to meet deadline. /------------------- | Step 70 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.97) Agent not enforced to meet deadline. /------------------- | Step 71 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.09) Agent not enforced to meet deadline. /------------------- | Step 72 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.15) Agent not enforced to meet deadline. /------------------- | Step 73 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.60) Agent not enforced to meet deadline. /------------------- | Step 74 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.92) Agent not enforced to meet deadline. /------------------- | Step 75 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.95) Agent not enforced to meet deadline. /------------------- | Step 76 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.34) Agent not enforced to meet deadline. /------------------- | Step 77 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.51) Agent not enforced to meet deadline. /------------------- | Step 78 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.93) Agent not enforced to meet deadline. /------------------- | Step 79 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.99) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Training trial 18 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.44) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.57) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.10) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.95) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.56) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.68) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.35) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.45) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.53) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.40) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.04) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.42) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.85) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.54) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.88) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.07) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.29) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.38) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.19) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.34) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.43) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.27) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.11) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.42) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.78) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.98) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.93) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.35) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.20) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.61) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.67) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.22) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.42) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.70) Agent not enforced to meet deadline. /------------------- | Step 34 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.88) Agent not enforced to meet deadline. /------------------- | Step 35 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.67) Agent not enforced to meet deadline. /------------------- | Step 36 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.96) Agent not enforced to meet deadline. /------------------- | Step 37 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.44) Agent not enforced to meet deadline. /------------------- | Step 38 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.83) Agent not enforced to meet deadline. /------------------- | Step 39 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.74) Agent not enforced to meet deadline. /------------------- | Step 40 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.27) Agent not enforced to meet deadline. /------------------- | Step 41 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.71) Agent not enforced to meet deadline. /------------------- | Step 42 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.73) Agent not enforced to meet deadline. /------------------- | Step 43 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.50) Agent not enforced to meet deadline. /------------------- | Step 44 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.06) Agent not enforced to meet deadline. /------------------- | Step 45 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.35) Agent not enforced to meet deadline. /------------------- | Step 46 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.30) Agent not enforced to meet deadline. /------------------- | Step 47 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.95) Agent not enforced to meet deadline. /------------------- | Step 48 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.19) Agent not enforced to meet deadline. /------------------- | Step 49 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.35) Agent not enforced to meet deadline. /------------------- | Step 50 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.10) Agent not enforced to meet deadline. /------------------- | Step 51 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.63) Agent not enforced to meet deadline. /------------------- | Step 52 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.72) Agent not enforced to meet deadline. /------------------- | Step 53 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.00) Agent not enforced to meet deadline. /------------------- | Step 54 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.59) Agent not enforced to meet deadline. /------------------- | Step 55 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.20) Agent not enforced to meet deadline. /------------------- | Step 56 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.27) Agent not enforced to meet deadline. /------------------- | Step 57 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.81) Agent not enforced to meet deadline. /------------------- | Step 58 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.54) Agent not enforced to meet deadline. /------------------- | Step 59 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.07) Agent not enforced to meet deadline. /------------------- | Step 60 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.78) Agent not enforced to meet deadline. /------------------- | Step 61 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.77) Agent not enforced to meet deadline. /------------------- | Step 62 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.57) Agent not enforced to meet deadline. /------------------- | Step 63 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.05) Agent not enforced to meet deadline. /------------------- | Step 64 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.10) Agent not enforced to meet deadline. /------------------- | Step 65 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.53) Agent not enforced to meet deadline. /------------------- | Step 66 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.62) Agent not enforced to meet deadline. /------------------- | Step 67 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.33) Agent not enforced to meet deadline. /------------------- | Step 68 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.99) Agent not enforced to meet deadline. /------------------- | Step 69 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.96) Agent not enforced to meet deadline. /------------------- | Step 70 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.50) Agent not enforced to meet deadline. /------------------- | Step 71 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.62) Agent not enforced to meet deadline. /------------------- | Step 72 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.74) Agent not enforced to meet deadline. /------------------- | Step 73 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.93) Agent not enforced to meet deadline. /------------------- | Step 74 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.50) Agent not enforced to meet deadline. /------------------- | Step 75 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.58) Agent not enforced to meet deadline. /------------------- | Step 76 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.58) Agent not enforced to meet deadline. /------------------- | Step 77 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.34) Agent not enforced to meet deadline. /------------------- | Step 78 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.52) Agent not enforced to meet deadline. /------------------- | Step 79 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.46) Agent not enforced to meet deadline. /------------------- | Step 80 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 1.93) Agent not enforced to meet deadline. /------------------- | Step 81 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.86) Agent not enforced to meet deadline. /------------------- | Step 82 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.66) Agent not enforced to meet deadline. /------------------- | Step 83 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.15) Agent not enforced to meet deadline. /------------------- | Step 84 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.45) Agent not enforced to meet deadline. /------------------- | Step 85 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.89) Agent not enforced to meet deadline. /------------------- | Step 86 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.37) Agent not enforced to meet deadline. /------------------- | Step 87 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.56) Agent not enforced to meet deadline. /------------------- | Step 88 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.10) Agent not enforced to meet deadline. /------------------- | Step 89 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.74) Agent not enforced to meet deadline. /------------------- | Step 90 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.64) Agent not enforced to meet deadline. /------------------- | Step 91 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.53) Agent not enforced to meet deadline. /------------------- | Step 92 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.76) Agent not enforced to meet deadline. /------------------- | Step 93 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.85) Agent not enforced to meet deadline. /------------------- | Step 94 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.75) Agent not enforced to meet deadline. /------------------- | Step 95 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.69) Agent not enforced to meet deadline. /------------------- | Step 96 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.44) Agent not enforced to meet deadline. /------------------- | Step 97 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.79) Agent not enforced to meet deadline. /------------------- | Step 98 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.16) Agent not enforced to meet deadline. /------------------- | Step 99 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.04) Agent not enforced to meet deadline. /------------------- | Step 100 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.98) Agent not enforced to meet deadline. /------------------- | Step 101 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.34) Agent not enforced to meet deadline. /------------------- | Step 102 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.95) Agent not enforced to meet deadline. /------------------- | Step 103 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.76) Agent not enforced to meet deadline. /------------------- | Step 104 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.96) Agent not enforced to meet deadline. /------------------- | Step 105 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.51) Agent not enforced to meet deadline. /------------------- | Step 106 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.35) Agent not enforced to meet deadline. /------------------- | Step 107 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.33) Agent not enforced to meet deadline. /------------------- | Step 108 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.07) Agent not enforced to meet deadline. /------------------- | Step 109 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.92) Agent not enforced to meet deadline. /------------------- | Step 110 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.12) Agent not enforced to meet deadline. /------------------- | Step 111 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.76) Agent not enforced to meet deadline. /------------------- | Step 112 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -11.00) Agent not enforced to meet deadline. /------------------- | Step 113 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.41) Agent not enforced to meet deadline. /------------------- | Step 114 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.06) Agent not enforced to meet deadline. /------------------- | Step 115 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.95) Agent not enforced to meet deadline. /------------------- | Step 116 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.85) Agent not enforced to meet deadline. /------------------- | Step 117 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.21) Agent not enforced to meet deadline. /------------------- | Step 118 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.29) Agent not enforced to meet deadline. /------------------- | Step 119 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.25) Agent not enforced to meet deadline. Trial Aborted! Agent did not reach the destination. /------------------------- | Training trial 19 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.23) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.31) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.47) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.89) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.06) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.18) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.02) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.05) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.67) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.03) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.73) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.05) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.06) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.54) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.47) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.16) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.52) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.66) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.31) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.68) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.40) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.73) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.77) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.88) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.40) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.47) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.40) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.84) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.60) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.20) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.52) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.91) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.20) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.96) Agent not enforced to meet deadline. /------------------- | Step 34 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.71) Agent not enforced to meet deadline. /------------------- | Step 35 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.01) Agent not enforced to meet deadline. /------------------- | Step 36 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.71) Agent not enforced to meet deadline. /------------------- | Step 37 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.20) Agent not enforced to meet deadline. /------------------- | Step 38 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.36) Agent not enforced to meet deadline. /------------------- | Step 39 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.74) Agent not enforced to meet deadline. /------------------- | Step 40 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.61) Agent not enforced to meet deadline. /------------------- | Step 41 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.54) Agent not enforced to meet deadline. /------------------- | Step 42 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.81) Agent not enforced to meet deadline. /------------------- | Step 43 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.38) Agent not enforced to meet deadline. /------------------- | Step 44 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.07) Agent not enforced to meet deadline. /------------------- | Step 45 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.36) Agent not enforced to meet deadline. /------------------- | Step 46 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.10) Agent not enforced to meet deadline. /------------------- | Step 47 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.58) Agent not enforced to meet deadline. /------------------- | Step 48 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.95) Agent not enforced to meet deadline. /------------------- | Step 49 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.32) Agent not enforced to meet deadline. /------------------- | Step 50 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.48) Agent not enforced to meet deadline. /------------------- | Step 51 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.97) Agent not enforced to meet deadline. /------------------- | Step 52 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.92) Agent not enforced to meet deadline. /------------------- | Step 53 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.03) Agent not enforced to meet deadline. /------------------- | Step 54 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.82) Agent not enforced to meet deadline. /------------------- | Step 55 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.48) Agent not enforced to meet deadline. /------------------- | Step 56 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.63) Agent not enforced to meet deadline. /------------------- | Step 57 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.43) Agent not enforced to meet deadline. /------------------- | Step 58 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.30) Agent not enforced to meet deadline. /------------------- | Step 59 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.83) Agent not enforced to meet deadline. /------------------- | Step 60 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.65) Agent not enforced to meet deadline. /------------------- | Step 61 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.08) Agent not enforced to meet deadline. /------------------- | Step 62 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.73) Agent not enforced to meet deadline. /------------------- | Step 63 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.85) Agent not enforced to meet deadline. /------------------- | Step 64 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.24) Agent not enforced to meet deadline. /------------------- | Step 65 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.69) Agent not enforced to meet deadline. /------------------- | Step 66 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.29) Agent not enforced to meet deadline. /------------------- | Step 67 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.58) Agent not enforced to meet deadline. /------------------- | Step 68 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.05) Agent not enforced to meet deadline. /------------------- | Step 69 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.78) Agent not enforced to meet deadline. /------------------- | Step 70 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.05) Agent not enforced to meet deadline. /------------------- | Step 71 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.68) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Training trial 20 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.97) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.26) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.57) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.91) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.30) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.55) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.23) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.03) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.81) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.05) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.33) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.77) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.84) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.11) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.86) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.93) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.53) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.85) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.77) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.39) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.22) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.53) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.96) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 2.00) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.46) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.36) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.27) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.42) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.16) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.09) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.34) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.05) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.35) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.39) Agent not enforced to meet deadline. /------------------- | Step 34 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.65) Agent not enforced to meet deadline. /------------------- | Step 35 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.43) Agent not enforced to meet deadline. /------------------- | Step 36 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.58) Agent not enforced to meet deadline. /------------------- | Step 37 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 2.65) Agent not enforced to meet deadline. /------------------- | Step 38 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.75) Agent not enforced to meet deadline. /------------------- | Step 39 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.65) Agent not enforced to meet deadline. /------------------- | Step 40 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.28) Agent not enforced to meet deadline. /------------------- | Step 41 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.60) Agent not enforced to meet deadline. /------------------- | Step 42 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.91) Agent not enforced to meet deadline. /------------------- | Step 43 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.47) Agent not enforced to meet deadline. /------------------- | Step 44 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.47) Agent not enforced to meet deadline. /------------------- | Step 45 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.51) Agent not enforced to meet deadline. /------------------- | Step 46 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.50) Agent not enforced to meet deadline. /------------------- | Step 47 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.66) Agent not enforced to meet deadline. /------------------- | Step 48 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.10) Agent not enforced to meet deadline. /------------------- | Step 49 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.97) Agent not enforced to meet deadline. /------------------- | Step 50 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.13) Agent not enforced to meet deadline. /------------------- | Step 51 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.63) Agent not enforced to meet deadline. /------------------- | Step 52 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -6.00) Agent not enforced to meet deadline. /------------------- | Step 53 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.14) Agent not enforced to meet deadline. /------------------- | Step 54 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.34) Agent not enforced to meet deadline. /------------------- | Step 55 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.16) Agent not enforced to meet deadline. /------------------- | Step 56 Results \------------------- !! Agent state not been updated! Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.83) Agent not enforced to meet deadline. /------------------- | Step 57 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.48) Agent not enforced to meet deadline. /------------------- | Step 58 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.15) Agent not enforced to meet deadline. /------------------- | Step 59 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.07) Agent not enforced to meet deadline. /------------------- | Step 60 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.92) Agent not enforced to meet deadline. /------------------- | Step 61 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.18) Agent not enforced to meet deadline. /------------------- | Step 62 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.68) Agent not enforced to meet deadline. /------------------- | Step 63 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.02) Agent not enforced to meet deadline. /------------------- | Step 64 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.37) Agent not enforced to meet deadline. /------------------- | Step 65 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.23) Agent not enforced to meet deadline. /------------------- | Step 66 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.62) Agent not enforced to meet deadline. /------------------- | Step 67 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.10) Agent not enforced to meet deadline. /------------------- | Step 68 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.44) Agent not enforced to meet deadline. /------------------- | Step 69 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.63) Agent not enforced to meet deadline. /------------------- | Step 70 Results \------------------- !! Agent state not been updated! Agent idled at a green light with oncoming traffic. (rewarded 0.67) Agent not enforced to meet deadline. /------------------- | Step 71 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.03) Agent not enforced to meet deadline. /------------------- | Step 72 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.76) Agent not enforced to meet deadline. /------------------- | Step 73 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.26) Agent not enforced to meet deadline. /------------------- | Step 74 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -9.86) Agent not enforced to meet deadline. /------------------- | Step 75 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.57) Agent not enforced to meet deadline. /------------------- | Step 76 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.20) Agent not enforced to meet deadline. /------------------- | Step 77 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.56) Agent not enforced to meet deadline. /------------------- | Step 78 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.50) Agent not enforced to meet deadline. /------------------- | Step 79 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.68) Agent not enforced to meet deadline. /------------------- | Step 80 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.07) Agent not enforced to meet deadline. /------------------- | Step 81 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.38) Agent not enforced to meet deadline. /------------------- | Step 82 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.51) Agent not enforced to meet deadline. /------------------- | Step 83 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 1.76) Agent not enforced to meet deadline. /------------------- | Step 84 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.60) Agent not enforced to meet deadline. /------------------- | Step 85 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.64) Agent not enforced to meet deadline. /------------------- | Step 86 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.43) Agent not enforced to meet deadline. /------------------- | Step 87 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.69) Agent not enforced to meet deadline. /------------------- | Step 88 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.35) Agent not enforced to meet deadline. /------------------- | Step 89 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.66) Agent not enforced to meet deadline. /------------------- | Step 90 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.62) Agent not enforced to meet deadline. /------------------- | Step 91 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.49) Agent not enforced to meet deadline. /------------------- | Step 92 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.03) Agent not enforced to meet deadline. /------------------- | Step 93 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 0.83) Agent not enforced to meet deadline. /------------------- | Step 94 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.67) Agent not enforced to meet deadline. /------------------- | Step 95 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.26) Agent not enforced to meet deadline. /------------------- | Step 96 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.94) Agent not enforced to meet deadline. /------------------- | Step 97 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.89) Agent not enforced to meet deadline. /------------------- | Step 98 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.24) Agent not enforced to meet deadline. /------------------- | Step 99 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -5.06) Agent not enforced to meet deadline. /------------------- | Step 100 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.19) Agent not enforced to meet deadline. /------------------- | Step 101 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 1.23) Agent not enforced to meet deadline. /------------------- | Step 102 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light. (rewarded -10.60) Agent not enforced to meet deadline. /------------------- | Step 103 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.02) Agent not enforced to meet deadline. /------------------- | Step 104 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.88) Agent not enforced to meet deadline. /------------------- | Step 105 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.63) Agent not enforced to meet deadline. Trial Completed! Agent reached the destination. /------------------------- | Testing trial 1 \------------------------- Simulating trial. . . Agent not set to learn. /------------------- | Step 0 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.06) Agent not enforced to meet deadline. /------------------- | Step 1 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.57) Agent not enforced to meet deadline. /------------------- | Step 2 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.70) Agent not enforced to meet deadline. /------------------- | Step 3 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.99) Agent not enforced to meet deadline. /------------------- | Step 4 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.42) Agent not enforced to meet deadline. /------------------- | Step 5 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 0.50) Agent not enforced to meet deadline. /------------------- | Step 6 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 0.57) Agent not enforced to meet deadline. /------------------- | Step 7 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.81) Agent not enforced to meet deadline. /------------------- | Step 8 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.78) Agent not enforced to meet deadline. /------------------- | Step 9 Results \------------------- !! Agent state not been updated! Agent drove forward instead of left. (rewarded 0.08) Agent not enforced to meet deadline. /------------------- | Step 10 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -9.19) Agent not enforced to meet deadline. /------------------- | Step 11 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.53) Agent not enforced to meet deadline. /------------------- | Step 12 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.45) Agent not enforced to meet deadline. /------------------- | Step 13 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.30) Agent not enforced to meet deadline. /------------------- | Step 14 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.17) Agent not enforced to meet deadline. /------------------- | Step 15 Results \------------------- !! Agent state not been updated! Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.80) Agent not enforced to meet deadline. /------------------- | Step 16 Results \------------------- !! Agent state not been updated! Agent idled at a green light with no oncoming traffic. (rewarded -4.82) Agent not enforced to meet deadline. /------------------- | Step 17 Results \------------------- !! Agent state not been updated! Agent properly idled at a red light. (rewarded 2.78) Agent not enforced to meet deadline. /------------------- | Step 18 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.41) Agent not enforced to meet deadline. /------------------- | Step 19 Results \------------------- !! Agent state not been updated! Agent drove right instead of left. (rewarded 1.02) Agent not enforced to meet deadline. /------------------- | Step 20 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 2.68) Agent not enforced to meet deadline. /------------------- | Step 21 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 1.38) Agent not enforced to meet deadline. /------------------- | Step 22 Results \------------------- !! Agent state not been updated! Agent drove forward instead of right. (rewarded 0.20) Agent not enforced to meet deadline. /------------------- | Step 23 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 1.72) Agent not enforced to meet deadline. /------------------- | Step 24 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.74) Agent not enforced to meet deadline. /------------------- | Step 25 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 2.03) Agent not enforced to meet deadline. /------------------- | Step 26 Results \------------------- !! Agent state not been updated! Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.54) Agent not enforced to meet deadline. /------------------- | Step 27 Results \------------------- !! Agent state not been updated! Agent drove right instead of forward. (rewarded 1.31) Agent not enforced to meet deadline. /------------------- | Step 28 Results \------------------- !! Agent state not been updated! Agent followed the waypoint left. (rewarded 1.83) Agent not enforced to meet deadline. /------------------- | Step 29 Results \------------------- !! Agent state not been updated! Agent followed the waypoint forward. (rewarded 1.33) Agent not enforced to meet deadline. /------------------- | Step 30 Results \------------------- !! Agent state not been updated! Agent followed the waypoint right. (rewarded 1.67) Agent not enforced to meet deadline. /------------------- | Step 31 Results \------------------- !! Agent state not been updated! Agent drove left instead of forward. (rewarded 1.82) Agent not enforced to meet deadline. /------------------- | Step 32 Results \------------------- !! Agent state not been updated! Agent drove left instead of right. (rewarded 0.97) Agent not enforced to meet deadline. /------------------- | Step 33 Results \------------------- !! Agent state not been updated! Agent attempted driving forward through a red light. (rewarded -10.80) Agent not enforced to meet deadline. Simulation ended. . .
问题 1¶
用几句话，描述在运行默认的agent.py代理程序中，你在模拟程序里观察到了什么。一些你可以考虑的情况：
- 在模拟过程中，智能出租车究竟移动了吗？
- 驾驶代理程序获得了什么样的奖励？
- 交通灯的颜色改变是如何影响奖励的？
提示： 从顶层的/smartcab/目录（这个notebook所在的地方），运行命令
'python smartcab/agent.py'
回答： 代理程序获得了从-5到+2一系列不同大小的奖励。模拟程序中出租车一直没有移动，但由于其所处的环境状态在每一步发生变化，所以每一个step中相当于代理程序都采取了no action，所以在交通灯变红时，代理程序会获得正数奖励，而变绿时奖励为负。
理解代码¶
除了要了解世界之外，还需要理解掌管世界、模拟程序等等如何运作的代码本身。如果一点也不去探索一下“隐藏”的器件，就试着去创建一个驾驶代理程序会很难。在顶层的/smartcab/的目录下，有两个文件夹：/logs/ （之后会用到）和/smartcab/。打开/smartcab/文件夹，探索每个下面的Python文件，然后回答下面的问题。
问题 2¶
- 在
agent.pyPython文件里，选择 3 个可以设定的 flag，并描述他们如何改变模拟程序的。 - 在
environment.pyPython文件里，当代理程序执行一个行动时，调用哪个Environment类的函数？ - 在
simulator.pyPython 文件里，'render_text()'函数和'render()'函数之间的区别是什么? - 在
planner.pyPython文件里，'next_waypoint()函数会先考虑南北方向还是东西方向?
回答：
- epsilon：如果设定为0.5，那么程序就会以0.5的概率选择一个随机的动作而不是通过Q-learning学习到的最佳动作。所以设定不同的值会影响程序对未知策略的探索程度。
- update_delay：设定每次刷新的时间间隔。
- alpha：为学习速率（learning rate），该参数在新计算出的Q值中调和了迭代计算中上一次的Q值，若设定为0.5则新计算的Q值中有一半是旧值，旧值占的比重越大则学习速度越慢（学习速率越小越细致吗？），旧值占比越小学习越快。
'act''render_text()'是在终端的文本性的输出，非图形界面；'render()'是在图形界面窗口渲染出小车在模拟路况上的动作状态。- 东西方向
Note¶
问题2-1被卡住
原因：某些flag项忘记了其概念定义，导致不理解。
解决：搜索了q-learning中的学习率α；巩固了对ε-greedy的理解。
实现一个基本的驾驶代理程序¶
创建一个优化Q-Learning的驾驶代理程序的第一步，是让代理程序确实地执行有效的行动。在这个情况下，一个有效的行动是None（不做任何行动）、'Left'（左转）、'Right'（右转）或者'Forward'（前进）。作为你的第一个实现，到'choose_action()'代理程序函数，使驾驶代理程序随机选择其中的一个动作。注意你会访问到几个类的成员变量，它们有助于你编写这个功能，比如'self.learning'和'self.valid_actions'。实现后，运行几次代理程序文件和模拟程序来确认你的驾驶代理程序每步都执行随机的动作。
Note¶
random.choice¶
random.choice从序列中获取一个随机元素。其函数原型为：random.choice(sequence)。参数sequence表示一个有序类型。这里要说明 一下：sequence在python不是一种特定的类型，而是泛指一系列的类型。list, tuple, 字符串都属于sequence。有关sequence可以查看python手册数据模型这一章。下面是使用choice的一些例子：
print random.choice("学习Python")
print random.choice(["JGood", "is", "a", "handsome", "boy"])
print random.choice(("Tuple", "List", "Dict"))基本代理程序模拟结果¶
要从最初的模拟程序获得结果，你需要调整下面的标志：
'enforce_deadline'- 将此标志设定为True来强制驾驶代理程序捕获它是否在合理时间内到达目的地。'update_delay'- 将此标志设定为较小数值（比如0.01）来减少每次试验中每步之间的时间。'log_metrics'- 将此标志设定为True将模拟结果记录为在/logs/目录下的.csv文件。'n_test'- 将此标志设定为'10'则执行10次测试试验。
可选的，你还可以通过将'display'标志设定为False来禁用可视化模拟（可以使得试验跑得更快）。调试时，设定的标志会返回到他们的默认设定。重要的是要理解每个标志以及它们如何影响到模拟。
你成功完成了最初的模拟后（有20个训练试验和10个测试试验），运行下面的代码单元格来使结果可视化。注意运行同样的模拟时，日志文件会被覆写，所以留意被载入的日志文件！在 projects/smartcab 下运行 agent.py 文件。
In [13]:
# Load the 'sim_no-learning' log file from the initial simulation results
vs.plot_trials('sim_no-learning.csv')
问题 3¶
利用上面的从你初始模拟中得到的可视化结果，给出关于驾驶代理程序的分析和若干观察。确保对于可视化结果上的每个面板你至少给出一条观察结果。你可以考虑的一些情况：
- 驾驶代理程序多频繁地做出不良决策？有多少不良决策造成了事故？
- 假定代理程序是随机驾驶，那么可靠率是否合理？
- 代理程序对于它的行动会获得什么样的奖励？奖励是否表明了它收到严重的惩罚？
- 随着试验数增加，结果输出是否有重大变化？
- 这个智能出租车对于乘客来说，会被人为是安全的且/或可靠的吗？为什么或者为什么不？
答案：
- 代理程序做不良决策的频率大约在34%-37%之间，大约1/3的不良决策造成了事故（包括轻微和严重事故）；
- 合理，从可靠性率图可以看出，可靠率很低，平均约为30％。因为模型是随机的，它没有经过任何训练。它也为一个模型设定了一个最低基准，即任何训练过的模型都应该比随机模型可靠率更高。
- 对于导致事故和违规的行动，代理程序会获得负数奖励，安全合理的行动会获得正数奖励。因为这个模型是一个随机模型，可以从可视化的结果中看到几乎每次行动奖励均为负数，表明这种这些随机的不合理的行动受到了严重惩罚。
- 没有，因为行动始终是随机产生的而没有参考训练经验，因此输出结果没有变得越来越好而是始终与随机行动的好坏一致。
- 不会，安全性可靠性都极低，行动也是随机的不合理的。
通知驾驶代理程序¶
创建一个优化Q-Learning的驾驶代理程序的第二步，是定义一系列代理程序会在环境中发生的状态。根据输入、感知数据和驾驶代理程序可用的变量，可以为代理程序定义一系列状态，使它最终可以学习在一个状态下它需要执行哪个动作。对于每个状态的'如果这个处于这个状态就那个行动'的状况称为策略，就是最终驾驶代理程序要学习的。没有定义状态，驾驶代理程序就不会明白哪个动作是最优的——或者甚至不会明白它要关注哪个环境变量和条件！
识别状态¶
查看'build_state()'代理程序函数，它显示驾驶代理函数可以从环境中获得下列数据：
'waypoint'，智能出租车去向目的地应该行驶的方向，它是智能出租车车头方向的相对值。'inputs'，智能出租车的感知器数据。它包括'light'，交通灯颜色。'left'，智能出租车左侧车辆的目的方向。如果没有车辆，则返回None。'right'，智能出租车右侧车辆的目的方向。如果没有车辆，则返回None。'oncoming'，智能出租车交叉方向车辆的目的方向。如果没有车辆，则返回None。
'deadline'，智能出租车在时间之内到达目的地还所需的剩余动作数目。
问题 4¶
代理程序的哪些可用特征与学习安全性和效率相关性最高？你为什么认为这些特征适合在环境中对智能出租车建模？如果你没有选择某些特征，放弃他们的原因是什么？
回答：
- 与安全性相关性最高的是'light'交通灯颜色，因为闯红灯是最容易导致意外的因素；其次是'left''right''oncoming'三个方向来车的情况，涉及到训练智能车‘躲避’‘礼让’的策略提高安全性。与效率相关性最高的是'waypoint'，保证智能车始终朝着目的地的方向前进而不会绕路，学会选择最优路线提高效率。
- 有了前进方向、交通灯信号和车体四周的来车状况，就可以判断出正确、安全的行动策略，并且前进方向可以通过导航获得、交通灯和左右前方来车状况可以通过传感器获得，适合在环境中对智能车建模。
- 没有选择'deadline'是因为该特征可能会导致智能车为了在剩余动作内到达目的地而采取不安全的策略，安全性的优先级应高于效率。
定义状态空间¶
当定义一系列代理程序会处于的状态，必需考虑状态空间的大小。就是说，如果你期望驾驶代理程序针对每个状态都学习一个策略，你会需要对于每一个代理状态都有一个最优的动作。如果所有可能状态的数量非常大，最后会变成这样的状况，驾驶代理程序对于某些状态学不到如何行动，会导致未学习过的决策。例如，考虑用下面的特征定义智能出租车的状态的情况：
('is_raining', 'is_foggy', 'is_red_light', 'turn_left', 'no_traffic', 'previous_turn_left', 'time_of_day').
发生如(False, True, True, True, False, False, '3AM')的状态的频次如何？没有近乎无限数量的训练，很怀疑代理程序会学到一个合适的动作！
问题 5¶
如果用你在问题4中选择的特征来定义一个状态，状态空间的大小是多少？假定你了解环境以及它是如何模拟的，你觉得经过合理数量的训练之后，代理驾驶能学到一个较好的策略吗？（遇见绝大部分状态都能作出正确决策。） 提示： 考虑特征组合来计算状态的总数！
回答： 状态空间的大小为3*2*3*3*3=162
不能，不能保证162种状态都能在训练中出现且被学习到较好的策略。
更新驾驶代理程序的状态¶
要完成你的第二个实现，去到'build_state()'代理程序函数。根据你在问题4给出的判断，你现在要将'state'变量设定为包含所有Q-Learning所需特征的元组。确认你的驾驶代理程序通过运行代理程序文件和模拟会更新它的状态，注意状态是否显示了。如果用了可视化模拟，确认更新的状态和在模拟程序里看到的一致。
注意： 观察时记住重置模拟程序的标志到默认设定！
实现Q-Learning驾驶代理程序¶
创建一个优化Q-Learning的驾驶代理程序的第三步，是开始实现Q-Learning自身的功能。Q-Learning的概念相当直接：每个访问的状态，为所有可用的状态-行动配对在Q-table里创建一条记录。然后，当代理程序遇到一个状态并执行了一个动作，基于获得的奖励和设定的相互的更新规则，来更新关联的状态-动作配对的Q-value。当然，Q-Learning还带来其他的收益，如此我们可以让代理程序根据每个可能的状态-动作配对的Q-values，来为每个状态选择最佳动作。在这个项目里，你会实现一个衰减 $\epsilon$ -贪心 的Q-learning算法，不含折扣因子。遵从每个代理程序函数的TODO下的指导实现。
注意代理程序的属性self.Q是一个字典：这就是Q-table的构成。每个状态是self.Q字典的键，每个值是另一个字典，包含了action和Q-value。这里是个样例：
{ 'state-1': {
'action-1' : Qvalue-1,
'action-2' : Qvalue-2,
...
},
'state-2': {
'action-1' : Qvalue-1,
...
},
...
}此外，注意你要求利用一个衰减$\epsilon$（探索）因子。因此，随着试验的增加，$\epsilon$会向0减小。这是因为，代理程序会从它的行为中学习，然后根据习得的行为行动。而且当$\epsilon$达到特定阈值后（默认阈值为0.01），代理程序被以它所学到的东西来作检测。作为初始的Q-Learning实现，你将实现一个线性衰减$\epsilon$的函数。
Q-Learning模拟结果¶
要从最初的Q-learning程序获得结果，你需要调整下面的标志和设置：
'enforce_deadline'- 将此标志设定为True来强制驾驶代理程序捕获它是否在合理时间内到达目的地。'update_delay'- 将此标志设定为较小数值（比如0.01）来减少每次试验中每步之间的时间。'log_metrics'- 将此标志设定为True将模拟结果记录为在/logs/目录下的.csv文件，Q-table存为.txt文件。'n_test'- 将此标志设定为'10'则执行10次测试试验。'learning'- 将此标志设定为'True'来告诉驾驶代理使用你的Q-Learning实现。
此外，使用下面的$\epsilon$衰减函数：
$$ \epsilon_{t+1} = \epsilon_{t} - 0.05, \hspace{10px}\textrm{for trial number } t$$
如果你在实施时遇到困难，尝试把'verbose'标志设为True来调试。调试时，在这里设定的标志会返回到它们的默认设定。重要的是你要理解每个标志做什么并且解释它们怎么影响模拟！
当你成功完成初始的Q-Learning模拟程序后，运行下面代码单元格来使结果可视化。注意当相同的模拟运行时，log文件会被覆写，所以要留意载入的log文件！
In [ ]:
# Load the 'sim_default-learning' file from the default Q-Learning simulation
vs.plot_trials('sim_default-learning.csv')
问题 6¶
利用上面的从你默认的Q-Learning模拟中得到的可视化结果，像在问题3那样，给出关于驾驶代理程序的分析和若干观察。注意模拟程序应该也产生了Q-table存在一个文本文件中，可以帮到你观察代理程序的算法。你可以考虑的一些情况：
- 有没有观察到基本驾驶代理程序和默认的Q-Learning代理程序的相似之处？
- 在测试之前驾驶代理大约需要做多少训练试验？在给定的$\epsilon$ 容忍度下，这个数字是否合理？
- 你实现的$\epsilon$（探索因子）衰减函数是否准确地在参数面板中显示？
- 随着试验数增加，不良动作的数目是否减少?平均奖励是否增加？
- 与初始的驾驶代理程序相比，安全性和可靠性评分怎样？
回答：
改进Q-Learning驾驶代理程序¶
创建一个优化Q-Learning的驾驶代理程序的第三步，是执行优化！现在Q-Learning算法已经实现并且驾驶代理程序已经成功学习了，需要调整设定、调节参数让驾驶代理程序学习安全性和效率。通常这一步需要很多试验和错误，因为某些设定必定会造成更糟糕的学习。要记住的一件事是学习的行为本身和需要的时间：理论上，我们可以允许代理程序用非常非常长的时间来学习；然而，Q-Learning另一个目的是将没有习得行为的试验试验变为有习得行为的行动。例如，训练中总让代理程序执行随机动作（如果$\epsilon = 1$并且永不衰减）当然可以使它学习，但是不会让它行动。当改进你的Q-Learning实现时，要考虑做一个特定的调整的意义，以及它是否逻辑上是否合理。
改进Q-Learning的模拟结果¶
要从最初的Q-learning程序获得结果，你需要调整下面的标志和设置：
'enforce_deadline'- 将此标志设定为True来强制驾驶代理程序捕获它是否在合理时间内到达目的地。'update_delay'- 将此标志设定为较小数值（比如0.01）来减少每次试验中每步之间的时间。'log_metrics'- 将此标志设定为True将模拟结果记录为在/logs/目录下的.csv文件，Q-table存为.txt文件。'learning'- 将此标志设定为'True'来告诉驾驶代理使用你的Q-Learning实现。'optimized'- 将此标志设定为'True'来告诉驾驶代理你在执行一个优化版本的Q-Learning实现。
优化Q-Learning代理程序可以调整的额外的标志：
'n_test'- 将此标志设定为某个正数（之前是10）来执行那么多次测试试验。'alpha'- 将此标志设定为0 - 1之间的实数来调整Q-Learning算法的学习率。'epsilon'- 将此标志设定为0 - 1之间的实数来调整Q-Learning算法的起始探索因子。'tolerance'- 将此标志设定为某个较小的大于0的值（默认是0.05）来设定测试的epsilon阈值。
此外，使用一个你选择的$\epsilon$ (探索因子)衰减函数。注意无论你用哪个函数，一定要以合理的速率衰减到'tolerance'。Q-Learning代理程序到此才可以开始测试。某个衰减函数的例子($t$是试验的数目)：
$$ \epsilon = a^t, \textrm{for } 0 < a < 1 \hspace{50px}\epsilon = \frac{1}{t^2}\hspace{50px}\epsilon = e^{-at}, \textrm{for } 0 < a < 1 \hspace{50px} \epsilon = \cos(at), \textrm{for } 0 < a < 1$$
如果你想的话，你也可以使用$\alpha$ （学习率） 的衰减函数，当然这通常比较少见。如果你这么做了，确保它满足不等式$0 \leq \alpha \leq 1$。
如果你在实施时遇到困难，尝试把'verbose'标志设为True来调试。调试时，在这里设定的标志会返回到它们的默认设定。重要的是你要理解每个标志做什么并且解释它们怎么影响模拟！
当你成功完成初始的Q-Learning模拟程序后，运行下面代码单元格来使结果可视化，请注意为了达到项目要求你需要在安全性和可靠性上获得至少都为A的评分。注意当相同的模拟运行时，log文件会被覆写，所以要留意载入的log文件！
In [20]:
%run smartcab\agent.py
#可以观察到可视化程序的各部分的功能
/-------------------------
| Training trial 1
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (4, 6), deadline = 30
Simulating trial. . .
epsilon = 0.9500; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: 1.65230188931
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.6523018893128723, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent drove right instead of forward. (rewarded 1.65)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: -4.48524769754
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 29, 't': 1, 'action': None, 'reward': -4.48524769754497, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.49)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: forward, reward: 1.09745481749
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 1.0974548174887437, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove forward instead of left. (rewarded 1.10)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: left, reward: -9.01789942037
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'left', 'reward': -9.017899420372219, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.02)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: forward, reward: 1.74737032069
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 1.7473703206911413, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.75)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: forward, reward: 0.72871843476
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 0.7287184347600949, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent drove forward instead of left. (rewarded 0.73)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 0.601458448518
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 0.6014584485180982, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 0.60)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 1.53703449645
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.537034496451263, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 1.54)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 1.84546473928
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 1.8454647392796204, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.85)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 0.489046632122
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 0.48904663212154076, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.49)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: forward, reward: -10.2477081564
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': -10.247708156424093, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.25)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: left, reward: 2.29271114593
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 2.292711145925718, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.29)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 2.22713458224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.2271345822377855, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.23)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: -20.0597357144
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 17, 't': 13, 'action': 'right', 'reward': -20.05973571444015, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.06)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 1.51565295151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.5156529515057342, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.52)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: 0.641286465401
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': 0.6412864654008946, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 0.64)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 1.0357592464
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 1.0357592463984473, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.04)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 0.00607040857004
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 13, 't': 17, 'action': None, 'reward': 0.006070408570035379, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.01)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 0.307058444215
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 12, 't': 18, 'action': 'forward', 'reward': 0.3070584442148896, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.31)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 2.31143164244
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 2.3114316424397137, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.31)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 2.03079018767
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 10, 't': 20, 'action': None, 'reward': 2.0307901876730114, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.03)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: forward, reward: -10.3378461101
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': -10.33784611007869, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.34)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: left, reward: -20.4526846505
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 8, 't': 22, 'action': 'left', 'reward': -20.452684650504906, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.45)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: forward, reward: 1.43374759899
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 7, 't': 23, 'action': 'forward', 'reward': 1.4337475989867983, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.43)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: -5.45556231731
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 6, 't': 24, 'action': None, 'reward': -5.455562317314192, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.46)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: forward, reward: 0.61259296268
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'left'), 'deadline': 5, 't': 25, 'action': 'forward', 'reward': 0.6125929626802702, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'left')
Agent drove forward instead of right. (rewarded 0.61)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 0.806641925311
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 4, 't': 26, 'action': None, 'reward': 0.8066419253112189, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.81)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: -19.7281363308
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', 'right'), 'deadline': 3, 't': 27, 'action': 'right', 'reward': -19.728136330818728, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', 'right')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.73)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: -4.77205713135
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'right', 'right', None), 'deadline': 2, 't': 28, 'action': None, 'reward': -4.772057131352998, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.77)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: forward, reward: 0.283267853678
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 1, 't': 29, 'action': 'forward', 'reward': 0.28326785367844876, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded 0.28)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 2
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (2, 6), deadline = 20
Simulating trial. . .
epsilon = 0.9000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 0.238047772421
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 0.23804777242103592, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.24)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: forward, reward: 0.132016677197
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 0.13201667719734644, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent drove forward instead of right. (rewarded 0.13)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: right, reward: 2.01535317136
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.015353171359024, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.02)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: left, reward: -10.5631360453
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -10.56313604528327, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.56)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: right, reward: 1.05232788856
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.0523278885632041, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.05)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 2.48914430738
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.4891443073804, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.49)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 0.954897239067
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.9548972390673207, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent drove right instead of forward. (rewarded 0.95)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: -4.11063476485
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': -4.110634764851165, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.11)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 2.77717187705
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 2.7771718770535836, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.78)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: -20.459486524
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': -20.459486523956468, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.46)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: right, reward: 0.898659990814
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.8986599908143659, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.90)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: forward, reward: 0.208845528237
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.20884552823664593, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded 0.21)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: left, reward: 0.721955027434
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 0.7219550274342013, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.72)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: left, reward: 1.3541808617
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.3541808617011575, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove left instead of right. (rewarded 1.35)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: -4.00202047673
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', 'right', 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': -4.002020476728631, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.00)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: -9.24796470125
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -9.247964701253789, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -9.25)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 1.79482796477
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'forward'), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.7948279647711147, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.79)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: left, reward: 0.356483882522
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 0.35648388252169394, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.36)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 0.473928524204
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 0.47392852420413334, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 0.47)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: -0.4440717414
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': -0.4440717414004077, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent drove right instead of left. (rewarded -0.44)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 3
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (3, 5), deadline = 20
Simulating trial. . .
epsilon = 0.8500; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 1.13200323957
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.1320032395703608, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.13)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: -10.7426102204
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -10.74261022043291, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.74)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: left, reward: -10.7162092378
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'left', 'reward': -10.716209237787341, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.72)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: right, reward: 2.51592874684
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 2.515928746843456, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.52)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: -0.0559452679356
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -0.05594526793562704, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent drove left instead of right. (rewarded -0.06)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: -20.8515483251
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': -20.85154832506655, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.85)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 2.26084571701
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'right'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.26084571701431, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'right')
Agent properly idled at a red light. (rewarded 2.26)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.736485052
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.7364850520046597, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.74)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: forward, reward: -10.9456058596
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -10.945605859639272, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.95)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: -4.87724663129
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': None, 'reward': -4.877246631285488, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.88)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: right, reward: 0.760374108841
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'forward'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.7603741088406231, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'forward')
Agent followed the waypoint right. (rewarded 0.76)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: left, reward: -0.245412403541
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': -0.2454124035409071, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove left instead of right. (rewarded -0.25)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: forward, reward: -39.1626229882
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': -39.162622988152584, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.16)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 1.61370711191
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.6137071119068875, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent drove right instead of forward. (rewarded 1.61)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 0.86292086896
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.8629208689601879, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 0.86)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 0.788867717776
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.788867717776015, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.79)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: forward, reward: -0.32407981165
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': -0.3240798116502681, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded -0.32)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: -0.446489036185
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -0.4464890361848387, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded -0.45)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: 1.5136704232
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': 1.5136704232003735, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.51)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: -9.9626204027
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': -9.962620402697201, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent attempted driving left through a red light. (rewarded -9.96)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 4
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (5, 7), deadline = 25
Simulating trial. . .
epsilon = 0.8000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: -5.36669100415
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': -5.366691004148889, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.37)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: forward, reward: 0.791215109864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 0.7912151098635274, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove forward instead of right. (rewarded 0.79)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 2.29871017262
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.2987101726157295, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.30)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: -19.8173666904
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 22, 't': 3, 'action': 'right', 'reward': -19.81736669037036, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.82)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: left, reward: -9.16992400789
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 21, 't': 4, 'action': 'left', 'reward': -9.169924007888383, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -9.17)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 2.57493506568
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.5749350656840546, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.57)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 2.2249312334
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.2249312334025153, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.22)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.97986985142
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.9798698514152848, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.98)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: -10.0301328788
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 8, 'action': 'left', 'reward': -10.030132878828677, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.03)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: -40.0637901712
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': -40.063790171152114, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.06)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 2.43928786649
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.439287866493789, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.44)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: -19.7341908235
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': -19.73419082352459, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.73)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 0.0694998749056
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 0.06949987490560883, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.07)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: forward, reward: 1.10124767097
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 1.10124767097133, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.10)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: forward, reward: 0.577746056858
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 0.577746056858396, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove forward instead of left. (rewarded 0.58)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: -20.7533480287
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': -20.753348028695477, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.75)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: forward, reward: -10.6515082533
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': -10.651508253347448, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.65)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: None, reward: 1.13468948596
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 8, 't': 17, 'action': None, 'reward': 1.134689485963356, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.13)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: forward, reward: -0.411831616843
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'right'), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': -0.41183161684311087, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'right')
Agent drove forward instead of left. (rewarded -0.41)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: forward, reward: -40.8527306301
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': -40.85273063013446, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.85)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 1.17738594233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 1.177385942332791, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 1.18)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: -0.565944134937
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': -0.5659441349373205, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded -0.57)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 2.13552500142
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 3, 't': 22, 'action': None, 'reward': 2.1355250014175398, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.14)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 1.36883907315
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'forward'), 'deadline': 2, 't': 23, 'action': 'forward', 'reward': 1.3688390731470963, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.37)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 0.991382584616
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.9913825846156026, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.99)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 5
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (7, 3), deadline = 20
Simulating trial. . .
epsilon = 0.7500; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 1.73681626843
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.736816268432499, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded 1.74)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 2.22582359226
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.2258235922603014, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.23)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: left, reward: -10.5087317487
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'left', 'reward': -10.508731748719025, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.51)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: left, reward: -10.7348399444
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -10.734839944418306, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -10.73)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: left, reward: -9.06547719572
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -9.065477195718815, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.07)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 2.91232180563
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.9123218056272506, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.91)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: -10.0069996218
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': -10.006999621763907, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'left')
Agent attempted driving left through a red light. (rewarded -10.01)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: -9.61583059527
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': -9.615830595268879, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.62)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: -9.50339389053
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -9.503393890526763, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.50)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: -0.0592072782058
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': -0.05920727820580374, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded -0.06)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: -39.9746925351
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -39.97469253506755, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.97)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: -10.7900980741
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': -10.790098074092644, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.79)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 0.907593123238
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.9075931232384389, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.91)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: -5.880500767
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': -5.880500767002617, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.88)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: 1.80278257751
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 1.8027825775109416, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent followed the waypoint left. (rewarded 1.80)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 1.23647096981
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.2364709698139418, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 1.24)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 0.836589280176
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.8365892801761465, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 0.84)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.11769469978
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 1.1176946997754995, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 1.12)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: -4.38140851998
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 2, 't': 18, 'action': None, 'reward': -4.381408519983069, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.38)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 0.961602558434
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 0.9616025584343365, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 0.96)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 6
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (8, 2), deadline = 20
Simulating trial. . .
epsilon = 0.7000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 1.14153174042
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.1415317404157488, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.14)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.87263653348
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.8726365334839357, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.87)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.36679860837
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.3667986083680694, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.37)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.97979573336
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.9797957333566598, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.98)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.96689713702
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.9668971370201356, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.97)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.44744147265
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.4474414726517333, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.45)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 1.53084551272
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.5308455127218261, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent drove right instead of forward. (rewarded 1.53)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 2.72074299323
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.720742993234742, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.72)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 0.977002610389
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 0.9770026103890712, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.98)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: forward, reward: 1.64527037547
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.6452703754722215, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.65)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 1.19711845967
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.1971184596690008, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 1.20)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: forward, reward: -10.0142186927
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': -10.014218692684468, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -10.01)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.33176325925
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.331763259253032, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.33)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 0.710197217364
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.7101972173641727, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.71)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: -5.21296262743
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 6, 't': 14, 'action': None, 'reward': -5.212962627425136, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.21)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: left, reward: -40.9896697775
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'forward'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -40.98966977753911, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.99)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: -10.1590568851
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': -10.159056885056764, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -10.16)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: left, reward: 1.23529084072
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', 'left'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 1.235290840723973, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', 'left')
Agent drove left instead of right. (rewarded 1.24)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 0.576542304964
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.5765423049635856, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.58)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 0.848769315832
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.8487693158315126, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.85)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 7
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (1, 4), deadline = 20
Simulating trial. . .
epsilon = 0.6500; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 1.93359980503
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.9335998050306902, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent drove right instead of left. (rewarded 1.93)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 1.75085277555
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.750852775552558, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.75)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 1.21058084695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.2105808469465162, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.21)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 1.88578792663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.8857879266341306, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.89)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: left, reward: 2.39711514683
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 2.3971151468262453, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.40)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: -5.05246423209
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': -5.052464232087496, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.05)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: 0.189504493472
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 0.18950449347243659, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.19)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: forward, reward: 1.65650895333
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.6565089533278878, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove forward instead of left. (rewarded 1.66)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: -5.01028600073
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 12, 't': 8, 'action': None, 'reward': -5.010286000726177, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.01)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: left, reward: -20.1566187304
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': -20.156618730433912, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.16)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: forward, reward: 1.74002245305
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.7400224530526316, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove forward instead of left. (rewarded 1.74)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: forward, reward: 1.41879687562
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.4187968756202265, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.42)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: 0.162151732915
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 0.16215173291485718, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent drove forward instead of left. (rewarded 0.16)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: -4.80759537199
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': -4.807595371992507, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.81)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: -5.68462025741
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 6, 't': 14, 'action': None, 'reward': -5.684620257413156, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.68)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: 2.43334107795
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 2.433341077946384, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.43)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: -9.71045574396
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': -9.71045574396475, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.71)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: -0.621413823483
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': None, 'reward': -0.6214138234830447, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.62)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: -0.627051137747
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': -0.6270511377465051, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent drove right instead of forward. (rewarded -0.63)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: -10.0281556743
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': -10.028155674279061, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.03)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 8
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (6, 2), deadline = 20
Simulating trial. . .
epsilon = 0.6000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 2.57488047393
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'right'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.5748804739273305, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'right')
Agent followed the waypoint forward. (rewarded 2.57)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: -9.28736773658
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -9.287367736578885, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.29)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: left, reward: -9.78954344185
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': 'left', 'reward': -9.789543441849984, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.79)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: left, reward: -9.39725874807
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -9.3972587480714, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.40)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 1.18621375967
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.1862137596735882, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.19)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 2.90453772687
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.9045377268670376, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.90)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: 2.29689980019
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 2.2968998001927474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.30)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: left, reward: 1.8377832685
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.8377832684955, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove left instead of forward. (rewarded 1.84)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: right, reward: 2.22561776996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.225617769961354, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 2.23)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 9
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (3, 6), deadline = 30
Simulating trial. . .
epsilon = 0.5500; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 1.6562771957
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.6562771957039677, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', 'left')
Agent drove right instead of forward. (rewarded 1.66)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: 1.81658055144
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.816580551438419, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.82)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.4769641567
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.4769641566991376, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.48)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: 0.69444587507
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 0.6944458750703398, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.69)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: left, reward: -10.8408098094
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 26, 't': 4, 'action': 'left', 'reward': -10.84080980943216, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.84)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 1.41023801014
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.4102380101399714, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.41)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: forward, reward: 0.230820382466
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 0.23082038246638514, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.23)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 1.64161648222
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.6416164822160755, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 1.64)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: -0.0407085920895
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 22, 't': 8, 'action': 'right', 'reward': -0.04070859208954858, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded -0.04)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: forward, reward: 2.67108936293
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 2.671089362934428, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.67)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: left, reward: -9.05119377692
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': 'left', 'reward': -9.051193776918355, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.05)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: 1.58152501247
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 1.5815250124694855, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.58)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 1.67662467285
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 1.6766246728496532, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 1.68)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: right, reward: 0.640511233978
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 0.6405112339780463, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove right instead of left. (rewarded 0.64)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: -5.49712212966
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', 'right'), 'deadline': 16, 't': 14, 'action': None, 'reward': -5.4971221296556685, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.50)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: -4.33541086101
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 15, 't': 15, 'action': None, 'reward': -4.335410861013193, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.34)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: forward, reward: 0.784077236838
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': 0.7840772368384146, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove forward instead of right. (rewarded 0.78)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: 1.59224355228
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.592243552276859, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.59)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: right, reward: -19.858891147
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 12, 't': 18, 'action': 'right', 'reward': -19.858891146987958, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.86)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: forward, reward: 1.13511217913
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': 1.1351121791253873, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.14)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: None, reward: 1.51036979339
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 10, 't': 20, 'action': None, 'reward': 1.510369793394461, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.51)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: None, reward: -4.753423648
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', None), 'deadline': 9, 't': 21, 'action': None, 'reward': -4.753423647997542, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.75)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: forward, reward: 0.193488351161
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': 0.1934883511605594, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded 0.19)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 1.56379836775
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 1.5637983677518528, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.56)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: 0.755769232847
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 6, 't': 24, 'action': 'left', 'reward': 0.7557692328465526, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.76)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: forward, reward: -0.457040830178
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 5, 't': 25, 'action': 'forward', 'reward': -0.45704083017783415, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent drove forward instead of right. (rewarded -0.46)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 0.762651950373
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 4, 't': 26, 'action': 'right', 'reward': 0.7626519503727824, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.76)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: left, reward: -10.1483033462
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 3, 't': 27, 'action': 'left', 'reward': -10.148303346221624, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -10.15)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 0.428356316886
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 2, 't': 28, 'action': None, 'reward': 0.42835631688615194, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.43)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.4911668037
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 1, 't': 29, 'action': None, 'reward': 1.4911668037036967, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.49)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 10
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (5, 2), deadline = 25
Simulating trial. . .
epsilon = 0.5000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 1.96054348614
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.9605434861406217, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.96)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 1.43997611965
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.4399761196475473, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.44)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 1.34393487609
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.3439348760897074, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.34)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 0.155436953155
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 0.1554369531546329, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent drove right instead of forward. (rewarded 0.16)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 0.0918252441411
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 0.09182524414108639, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.09)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: left, reward: 1.09157029907
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.0915702990707712, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.09)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: -19.0211428594
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': -19.021142859360715, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.02)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: -0.07650287889
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': -0.07650287889002905, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent drove right instead of forward. (rewarded -0.08)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: 1.69520834435
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 1.6952083443502461, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.70)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: -10.1035109965
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': -10.103510996547516, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.10)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.00193159914
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.0019315991406534, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.00)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: -5.47024561506
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'forward'), 'deadline': 14, 't': 11, 'action': None, 'reward': -5.4702456150566166, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.47)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 0.486227209906
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 0.48622720990624857, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove right instead of forward. (rewarded 0.49)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: forward, reward: -9.75487220924
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': -9.75487220924435, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -9.75)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 0.989480629626
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 0.9894806296261983, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.99)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: -4.73059443762
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 10, 't': 15, 'action': None, 'reward': -4.730594437624731, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.73)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: left, reward: -39.7407933003
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'left'), 'deadline': 9, 't': 16, 'action': 'left', 'reward': -39.74079330027431, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.74)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: -10.4565082544
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': -10.45650825441799, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.46)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 2.31236673271
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 2.3123667327100588, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.31)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: left, reward: -39.3965782042
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 6, 't': 19, 'action': 'left', 'reward': -39.39657820422844, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.40)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: left, reward: -10.6770222644
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 5, 't': 20, 'action': 'left', 'reward': -10.677022264402135, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent attempted driving left through a red light. (rewarded -10.68)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 0.866239681903
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', 'left'), 'deadline': 4, 't': 21, 'action': None, 'reward': 0.8662396819028012, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.87)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 0.574743715987
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', 'left'), 'deadline': 3, 't': 22, 'action': None, 'reward': 0.5747437159867683, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.57)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: left, reward: 0.589125726455
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 2, 't': 23, 'action': 'left', 'reward': 0.5891257264550744, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove left instead of right. (rewarded 0.59)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: left, reward: 0.994870536973
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 1, 't': 24, 'action': 'left', 'reward': 0.994870536973236, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.99)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 11
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (1, 5), deadline = 20
Simulating trial. . .
epsilon = 0.4500; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 1.63203013679
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.6320301367929666, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.63)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: 1.09192603591
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'right'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.0919260359062144, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'right')
Agent drove forward instead of right. (rewarded 1.09)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 2.54592214572
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.5459221457171486, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.55)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 2.02926638326
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 2.029266383255065, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.03)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 0.896115267233
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.8961152672325784, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 0.90)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: left, reward: -10.8653722793
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': -10.86537227926906, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent attempted driving left through a red light. (rewarded -10.87)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: forward, reward: 1.55923278116
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.5592327811579676, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent drove forward instead of left. (rewarded 1.56)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 1.04322647166
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.0432264716632993, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.04)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: left, reward: -10.5465125806
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -10.546512580637941, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent attempted driving left through a red light. (rewarded -10.55)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: forward, reward: 1.20606763812
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.2060676381178221, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.21)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: left, reward: 0.835612957274
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 0.8356129572735747, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', None)
Agent followed the waypoint left. (rewarded 0.84)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 2.2626292003
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.2626292002957666, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.26)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 1.00054302363
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.000543023628005, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.00)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: forward, reward: 0.997193700289
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.9971937002892305, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.00)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: left, reward: -40.9200548447
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', None, 'right'), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -40.92005484474312, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.92)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 1.15773055862
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.1577305586194977, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 1.16)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: right, reward: 0.586701884304
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.5867018843039642, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 0.59)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: left, reward: 0.708718820701
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 0.7087188207013039, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove left instead of forward. (rewarded 0.71)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: -0.0725858680019
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': -0.07258586800187539, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded -0.07)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: forward, reward: -0.250036841354
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -0.25003684135437465, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded -0.25)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 12
\-------------------------
Environment.reset(): Trial set up with start = (7, 3), destination = (3, 7), deadline = 30
Simulating trial. . .
epsilon = 0.4000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: right, reward: 2.71073484246
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 2.7107348424629616, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.71)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: -9.65835674877
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': -9.658356748768938, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.66)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 2.55702918399
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.557029183985869, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.56)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: left, reward: -9.21884960817
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'left', 'reward': -9.218849608166208, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.22)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: left, reward: 1.27349709274
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 1.2734970927369842, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent drove left instead of forward. (rewarded 1.27)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: forward, reward: 1.58834062411
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.5883406241100424, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove forward instead of right. (rewarded 1.59)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: 0.971605298741
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 0.9716052987410487, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.97)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 2.59115481177
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 2.591154811774711, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.59)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 1.85547773242
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 1.855477732424218, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.86)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: -5.79599753
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 21, 't': 9, 'action': None, 'reward': -5.795997530004975, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.80)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 2.70269009031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.702690090310612, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.70)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: -10.7186189847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 19, 't': 11, 'action': 'left', 'reward': -10.718618984670911, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.72)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: -9.89959047391
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': -9.899590473910042, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.90)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: 1.57209528776
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 1.572095287761037, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.57)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 2.73404274934
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 16, 't': 14, 'action': None, 'reward': 2.7340427493362647, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.73)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.51094459307
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.5109445930744767, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.51)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: 1.43862599485
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 1.4386259948529025, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.44)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 1.08550194088
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.085501940879453, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.09)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 2.120664694
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 18, 'action': None, 'reward': 2.1206646939962273, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.12)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 2.17206968513
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 11, 't': 19, 'action': None, 'reward': 2.172069685127885, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.17)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 2.29322497403
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 10, 't': 20, 'action': None, 'reward': 2.2932249740325474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.29)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 1.02915448861
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 1.0291544886063937, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.03)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: 1.84296004265
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': 1.8429600426535044, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.84)
23% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 13
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (7, 3), deadline = 35
Simulating trial. . .
epsilon = 0.3500; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.51690081102
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 35, 't': 0, 'action': None, 'reward': 1.5169008110232247, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.52)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 1.747374316
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 34, 't': 1, 'action': 'right', 'reward': 1.7473743160012407, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 1.75)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 2.74071724886
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 33, 't': 2, 'action': None, 'reward': 2.7407172488587053, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.74)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 2.90733967736
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 32, 't': 3, 'action': None, 'reward': 2.90733967735838, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.91)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 0.297656891781
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 31, 't': 4, 'action': 'right', 'reward': 0.29765689178078203, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.30)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: forward, reward: 0.769028790896
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 30, 't': 5, 'action': 'forward', 'reward': 0.7690287908963174, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.77)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: left, reward: 1.64650439044
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 29, 't': 6, 'action': 'left', 'reward': 1.6465043904383783, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.65)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 0.883532625696
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 28, 't': 7, 'action': None, 'reward': 0.8835326256955761, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.88)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 2.82210325165
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 27, 't': 8, 'action': 'forward', 'reward': 2.8221032516546956, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.82)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.35814279448
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 26, 't': 9, 'action': None, 'reward': 2.358142794477501, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.36)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.0743467575
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 25, 't': 10, 'action': None, 'reward': 1.0743467574950378, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 1.07)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: forward, reward: 1.79568771546
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 24, 't': 11, 'action': 'forward', 'reward': 1.7956877154587045, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.80)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 0.949172306468
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 23, 't': 12, 'action': None, 'reward': 0.9491723064679685, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 0.95)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: 1.6706428116
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 22, 't': 13, 'action': 'right', 'reward': 1.6706428115961824, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.67)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 14
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (8, 2), deadline = 20
Simulating trial. . .
epsilon = 0.3000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: forward, reward: 2.03902606058
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'right'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.039026060576641, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'right')
Agent followed the waypoint forward. (rewarded 2.04)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: left, reward: 0.977800976482
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 0.9778009764821519, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove left instead of forward. (rewarded 0.98)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 2.14399840191
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.1439984019088962, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.14)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 2.57751584307
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.5775158430700715, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.58)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 1.16358451701
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.163584517008505, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.16)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: left, reward: 2.50955419143
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.5095541914266315, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 2.51)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 0.034161902433
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.03416190243304573, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.03)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: -5.41627008201
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 13, 't': 7, 'action': None, 'reward': -5.416270082009899, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.42)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: -5.83619090927
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 12, 't': 8, 'action': None, 'reward': -5.836190909267529, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.84)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: 2.32674196342
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 2.3267419634191753, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 2.33)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.27210685526
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.2721068552636279, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.27)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: left, reward: 2.71283717818
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 2.7128371781810134, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 2.71)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 15
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (1, 4), deadline = 30
Simulating trial. . .
epsilon = 0.2500; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: forward, reward: 1.93048713624
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': 1.9304871362402005, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.93)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: right, reward: 1.47501267482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.475012674815539, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent drove right instead of left. (rewarded 1.48)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: -9.56660186131
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 28, 't': 2, 'action': 'left', 'reward': -9.566601861307063, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent attempted driving left through a red light. (rewarded -9.57)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 1.196701123
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.196701123000162, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.20)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 0.141499248287
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 0.14149924828656812, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.14)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: 2.39693240765
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 2.3969324076470064, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.40)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.33885029639
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'right', 'left'), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.3388502963919628, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.34)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: 0.464086731449
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 0.4640867314485789, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.46)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: 0.69815988445
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 0.6981598844495785, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent drove right instead of left. (rewarded 0.70)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: -0.0459459843823
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 21, 't': 9, 'action': 'right', 'reward': -0.04594598438227804, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded -0.05)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: None, reward: -4.77047399278
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'right', 'forward'), 'deadline': 20, 't': 10, 'action': None, 'reward': -4.770473992777143, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.77)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: 1.15376587647
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.153765876469493, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.15)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 1.71351249571
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 1.7135124957051777, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.71)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.11399012972
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 13, 'action': None, 'reward': 2.1139901297151686, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.11)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: -19.9751679401
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', 'forward'), 'deadline': 16, 't': 14, 'action': 'right', 'reward': -19.97516794007063, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.98)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: -5.51861282934
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 15, 't': 15, 'action': None, 'reward': -5.518612829338508, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.52)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: left, reward: 0.1342029666
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 0.13420296659965003, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove left instead of forward. (rewarded 0.13)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 1.09018751989
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.0901875198923279, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.09)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 2.21912880148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 12, 't': 18, 'action': None, 'reward': 2.219128801483432, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.22)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: 1.48117769982
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': 1.4811776998190305, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.48)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 1.1044009397
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 1.1044009397002517, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.10)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.86556136677
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 21, 'action': None, 'reward': 1.8655613667665654, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.87)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: -9.90232840632
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', None), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': -9.902328406324465, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', None)
Agent attempted driving forward through a red light. (rewarded -9.90)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: left, reward: -0.0664881493748
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 7, 't': 23, 'action': 'left', 'reward': -0.06648814937477565, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove left instead of forward. (rewarded -0.07)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 0.779814459062
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 0.7798144590623772, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.78)
17% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 16
\-------------------------
Environment.reset(): Trial set up with start = (7, 3), destination = (1, 5), deadline = 20
Simulating trial. . .
epsilon = 0.2000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: -4.84783160724
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': -4.847831607235088, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.85)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: forward, reward: 1.19863824813
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.1986382481330655, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 1.20)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: forward, reward: 1.61293367673
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.612933676731303, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 1.61)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: 0.0310473275117
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 0.03104732751168926, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 0.03)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 0.849200247988
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.8492002479877084, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 0.85)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 1.43128330218
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.4312833021752132, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.43)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 2.84770649903
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.8477064990251977, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.85)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: -4.43840468159
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'right', 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': -4.438404681589466, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.44)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: left, reward: 1.67529696149
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.6752969614934057, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.68)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: left, reward: -40.2219745162
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': -40.22197451624338, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.22)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: left, reward: 0.443165912121
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 0.4431659121214083, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', 'left')
Agent drove left instead of forward. (rewarded 0.44)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: left, reward: 0.444023148822
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 0.4440231488221744, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.44)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: -4.99491324844
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': -4.994913248439244, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.99)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 1.85770231179
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.8577023117922917, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 1.86)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 2.38318241748
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.3831824174827867, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.38)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 1.20587446436
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.20587446436181, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.21)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: -39.0887816712
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': -39.08878167119372, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.09)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 0.551475377885
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.5514753778852621, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.55)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 1.38023348879
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.3802334887898144, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.38)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 1.39504894806
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.395048948055086, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.40)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 17
\-------------------------
Environment.reset(): Trial set up with start = (2, 2), destination = (7, 6), deadline = 25
Simulating trial. . .
epsilon = 0.1500; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: 2.89303608497
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.893036084972799, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.89)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: left, reward: 0.909557707574
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 0.9095577075743357, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 0.91)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 1.60289567969
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.6028956796918639, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.60)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: -9.09664432974
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': -9.096644329742531, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent attempted driving forward through a red light. (rewarded -9.10)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 2.1123128075
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.112312807495982, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.11)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 1.46915982358
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.4691598235787657, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.47)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 2.60212514438
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.6021251443816773, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.60)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 0.648238982705
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.6482389827052721, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.65)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 0.57724604206
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 0.5772460420601321, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove forward instead of left. (rewarded 0.58)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 0.883951882596
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': None, 'reward': 0.8839518825959005, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.88)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.04467184373
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.04467184373271, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.04)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: left, reward: -20.2919205367
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': -20.291920536668258, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.29)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 1.572047105
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 1.572047105003538, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 1.57)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 1.72614738111
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.726147381107255, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent drove right instead of left. (rewarded 1.73)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 0.860361509967
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 0.8603615099669888, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.86)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: forward, reward: 0.0989275232418
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 0.09892752324177412, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.10)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: forward, reward: -39.8496498757
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'left'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': -39.84964987569218, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.85)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 2.22203217148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 8, 't': 17, 'action': None, 'reward': 2.2220321714811755, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.22)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 2.22850312276
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 2.22850312275893, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.23)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: left, reward: 0.425724519655
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 0.425724519655334, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.43)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: -4.03678139591
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'forward'), 'deadline': 5, 't': 20, 'action': None, 'reward': -4.036781395908683, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.04)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: left, reward: -40.3338765402
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': -40.33387654022485, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.33)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 0.555714376828
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.5557143768281727, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.56)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 2.1447347179
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 2, 't': 23, 'action': None, 'reward': 2.1447347178984737, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.14)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: forward, reward: -9.49104981301
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, 'right'), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': -9.49104981300522, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'right', None, 'right')
Agent attempted driving forward through a red light. (rewarded -9.49)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 18
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (4, 2), deadline = 30
Simulating trial. . .
epsilon = 0.1000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 1.45718168217
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': 1.4571816821728447, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.46)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: left, reward: 0.00913378600949
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 29, 't': 1, 'action': 'left', 'reward': 0.009133786009485001, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.01)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: right, reward: 1.69492918996
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 1.6949291899561887, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.69)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: right, reward: 1.68431041036
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 1.6843104103593802, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.68)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: left, reward: 1.12893770731
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 1.1289377073065405, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.13)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: left, reward: 0.851459255971
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 0.8514592559707236, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove left instead of forward. (rewarded 0.85)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 2.04007691003
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 2.0400769100256833, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.04)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 1.53566183725
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.5356618372483748, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 1.54)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: forward, reward: 1.01055989275
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': 1.0105598927465378, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 1.01)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: forward, reward: 0.704243496043
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 0.7042434960427699, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.70)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: 2.06376973945
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.063769739453416, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.06)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: 1.07959457865
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.0795945786464292, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove forward instead of left. (rewarded 1.08)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: 1.12770920649
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 1.1277092064929986, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 1.13)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: -39.7198275083
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'right'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': -39.71982750825332, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'right')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.72)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 1.62054374577
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.620543745773001, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.62)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 2.2245554494
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 2.224555449400249, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.22)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.76155739767
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 16, 'action': None, 'reward': 1.761557397669904, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.76)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: 0.414142276734
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 0.4141422767337384, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded 0.41)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: 0.989334383455
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 0.989334383454769, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent drove right instead of left. (rewarded 0.99)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: forward, reward: 0.165152053542
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': 0.16515205354168982, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded 0.17)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 1.45778112312
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 1.4577811231221087, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.46)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: forward, reward: 0.118823780343
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 0.11882378034312213, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.12)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 0.914135709477
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 8, 't': 22, 'action': None, 'reward': 0.9141357094765235, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.91)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: left, reward: 0.618475216322
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 7, 't': 23, 'action': 'left', 'reward': 0.6184752163221038, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove left instead of right. (rewarded 0.62)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.52004421394
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': None, 'reward': 1.5200442139429686, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.52)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: left, reward: 0.358880632685
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 25, 'action': 'left', 'reward': 0.3588806326849585, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.36)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.46530027735
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 4, 't': 26, 'action': None, 'reward': 1.4653002773520776, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.47)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.43244106437
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 3, 't': 27, 'action': None, 'reward': 1.4324410643692176, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.43)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.88961632277
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 28, 'action': None, 'reward': 1.8896163227683986, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.89)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.89433897894
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 29, 'action': None, 'reward': 1.8943389789356286, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.89)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 19
\-------------------------
Environment.reset(): Trial set up with start = (7, 6), destination = (3, 3), deadline = 35
Simulating trial. . .
epsilon = 0.0500; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: forward, reward: 0.469999343555
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 35, 't': 0, 'action': 'forward', 'reward': 0.46999934355511486, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 0.47)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: 2.09091307494
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 34, 't': 1, 'action': 'left', 'reward': 2.090913074940322, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.09)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: 1.62659493834
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 33, 't': 2, 'action': 'left', 'reward': 1.626594938335945, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 1.63)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: -4.51839777258
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 32, 't': 3, 'action': None, 'reward': -4.518397772582835, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.52)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 1.40439262205
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 31, 't': 4, 'action': None, 'reward': 1.4043926220485996, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.40)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 2.46977691348
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 30, 't': 5, 'action': 'right', 'reward': 2.4697769134793117, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.47)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 0.35241447749
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 29, 't': 6, 'action': 'right', 'reward': 0.3524144774900029, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.35)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: forward, reward: 0.711012595725
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 28, 't': 7, 'action': 'forward', 'reward': 0.7110125957247956, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove forward instead of left. (rewarded 0.71)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 0.555297666235
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 27, 't': 8, 'action': 'right', 'reward': 0.5552976662353434, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent drove right instead of left. (rewarded 0.56)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 0.945067560158
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 26, 't': 9, 'action': None, 'reward': 0.9450675601578569, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.95)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 1.79046517047
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 25, 't': 10, 'action': 'forward', 'reward': 1.7904651704655863, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.79)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.50886703731
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 24, 't': 11, 'action': None, 'reward': 2.5088670373074633, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.51)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 0.862674903118
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 23, 't': 12, 'action': 'right', 'reward': 0.8626749031178297, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent drove right instead of left. (rewarded 0.86)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 2.68481942075
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 22, 't': 13, 'action': None, 'reward': 2.684819420749429, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.68)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 1.35709656846
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 21, 't': 14, 'action': 'right', 'reward': 1.3570965684644578, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.36)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: -4.79330370826
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 20, 't': 15, 'action': None, 'reward': -4.793303708257281, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.79)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 0.349781317916
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 19, 't': 16, 'action': 'right', 'reward': 0.3497813179160284, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.35)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 1.14094690007
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 17, 'action': 'right', 'reward': 1.140946900065349, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.14)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 0.738432151201
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'forward'), 'deadline': 17, 't': 18, 'action': 'right', 'reward': 0.7384321512011189, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'forward')
Agent drove right instead of left. (rewarded 0.74)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 0.982053039638
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 16, 't': 19, 'action': 'right', 'reward': 0.982053039638271, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 0.98)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.17230323294
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 15, 't': 20, 'action': None, 'reward': 1.1723032329356489, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.17)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: -20.6616072009
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'forward'), 'deadline': 14, 't': 21, 'action': 'left', 'reward': -20.661607200944378, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.66)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: -0.268019827257
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 13, 't': 22, 'action': 'left', 'reward': -0.2680198272570342, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove left instead of forward. (rewarded -0.27)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: forward, reward: -9.5987906048
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'right', None), 'deadline': 12, 't': 23, 'action': 'forward', 'reward': -9.598790604803026, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', None)
Agent attempted driving forward through a red light. (rewarded -9.60)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 2.28194732823
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 24, 'action': 'right', 'reward': 2.2819473282265053, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.28)
29% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: -20.2577997653
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 10, 't': 25, 'action': 'left', 'reward': -20.257799765262785, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.26)
26% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 1.63964390971
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 9, 't': 26, 'action': 'forward', 'reward': 1.6396439097143716, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.64)
23% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: -0.383625257011
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 8, 't': 27, 'action': 'right', 'reward': -0.38362525701083683, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded -0.38)
20% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: forward, reward: 0.129624747017
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 28, 'action': 'forward', 'reward': 0.12962474701706794, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.13)
17% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 1.17910188431
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 6, 't': 29, 'action': None, 'reward': 1.1791018843079752, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.18)
14% of time remaining to reach destination.
/-------------------
| Step 30 Results
\-------------------
Environment.step(): t = 30
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 1.65962911509
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 5, 't': 30, 'action': None, 'reward': 1.6596291150929638, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.66)
11% of time remaining to reach destination.
/-------------------
| Step 31 Results
\-------------------
Environment.step(): t = 31
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 0.672120477186
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 4, 't': 31, 'action': None, 'reward': 0.6721204771862235, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.67)
9% of time remaining to reach destination.
/-------------------
| Step 32 Results
\-------------------
Environment.step(): t = 32
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 1.46159283968
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 3, 't': 32, 'action': None, 'reward': 1.4615928396839397, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.46)
6% of time remaining to reach destination.
/-------------------
| Step 33 Results
\-------------------
Environment.step(): t = 33
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: left, reward: 0.435600196744
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 33, 'action': 'left', 'reward': 0.4356001967435148, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.44)
3% of time remaining to reach destination.
/-------------------
| Step 34 Results
\-------------------
Environment.step(): t = 34
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: -39.1554450383
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 1, 't': 34, 'action': 'forward', 'reward': -39.155445038296165, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.16)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 20
\-------------------------
Environment.reset(): Trial set up with start = (8, 7), destination = (2, 5), deadline = 20
Simulating trial. . .
epsilon = -0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 0.217172286897
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.21717228689706336, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent drove right instead of forward. (rewarded 0.22)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 1.64741844671
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.6474184467135493, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 1.65)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: left, reward: -9.66060304663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 18, 't': 2, 'action': 'left', 'reward': -9.660603046631316, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.66)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 1.25948233621
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.259482336214732, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.26)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 2.40606544345
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.4060654434472584, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.41)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 0.318782066653
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.318782066653466, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded 0.32)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: left, reward: 2.3636548822
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.363654882195206, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.36)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 1.72133859801
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.7213385980058848, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', None)
Agent followed the waypoint forward. (rewarded 1.72)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.14051911343
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', 'left'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.1405191134307016, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', 'left')
Agent drove forward instead of left. (rewarded 1.14)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.84906600634
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.8490660063366624, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.85)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: 1.76908427285
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.7690842728480918, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.77)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 0.784442126728
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 0.7844421267283359, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.78)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 1.4327045232
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.4327045232025712, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.43)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: forward, reward: -0.228903085063
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': -0.22890308506324664, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded -0.23)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: -0.395876442115
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': -0.39587644211508544, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove right instead of left. (rewarded -0.40)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 0.63734493855
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 0.6373449385501557, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 0.64)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 1.79160237232
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'left'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.791602372319665, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.79)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 0.984706676758
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.9847066767576556, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.98)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.1755138399
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 2.1755138399042737, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.18)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: -0.384218544364
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': -0.3842185443640892, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded -0.38)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 1
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (1, 7), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: forward, reward: 1.93375338727
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.933753387265525, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 1.93)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: right, reward: 0.115013420595
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.11501342059461739, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 0.12)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 1.15917980343
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.1591798034250953, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.16)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.12875672952
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.1287567295249987, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.13)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 0.0425368473886
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 0.04253684738857444, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.04)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 0.493900586865
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.493900586864796, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 0.49)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: 2.55546607566
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.5554660756612577, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 2.56)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.2088287502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.2088287502008574, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.21)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 0.994441316771
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.9944413167708298, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 0.99)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: -5.33200016649
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 16, 't': 9, 'action': None, 'reward': -5.332000166492403, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.33)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 0.808709962571
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.8087099625706341, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 0.81)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: right, reward: 1.03789140821
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.0378914082108672, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.04)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 1.7304959544
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.7304959544043466, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.73)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 0.684047900412
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 0.6840479004119064, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded 0.68)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: left, reward: 2.00886599057
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 2.0088659905709956, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.01)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: left, reward: 1.40985671966
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 1.4098567196645362, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove left instead of forward. (rewarded 1.41)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 1.62012223761
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 1.620122237612143, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.62)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: -9.33416135511
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 8, 't': 17, 'action': 'left', 'reward': -9.334161355105113, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.33)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 1.25211571129
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.2521157112855952, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.25)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: -5.91648173803
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', 'forward'), 'deadline': 6, 't': 19, 'action': None, 'reward': -5.916481738031304, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.92)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: -4.43800251161
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', 'right'), 'deadline': 5, 't': 20, 'action': None, 'reward': -4.43800251160856, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.44)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: -5.76680189506
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', 'forward'), 'deadline': 4, 't': 21, 'action': None, 'reward': -5.766801895064195, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.77)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: 0.988788980249
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.9887889802490748, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.99)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: right, reward: -0.508910289402
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'right'), 'deadline': 2, 't': 23, 'action': 'right', 'reward': -0.5089102894018565, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'right')
Agent drove right instead of left. (rewarded -0.51)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: -4.36895499409
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 1, 't': 24, 'action': None, 'reward': -4.368954994088281, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.37)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 2
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (5, 6), deadline = 30
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 1.98770608584
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.9877060858364284, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent followed the waypoint right. (rewarded 1.99)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: left, reward: 0.19682196748
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'left', 'left'), 'deadline': 29, 't': 1, 'action': 'left', 'reward': 0.19682196748025105, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'left', 'left')
Agent drove left instead of right. (rewarded 0.20)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: left, reward: 0.811794123218
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 28, 't': 2, 'action': 'left', 'reward': 0.8117941232179781, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove left instead of forward. (rewarded 0.81)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 2.16349733619
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 2.163497336185885, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.16)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 0.0848304258474
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 0.08483042584741673, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded 0.08)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: left, reward: 1.58845850419
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 1.5884585041882084, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.59)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.48495531156
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.4849553115626293, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.48)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 0.496794641532
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 0.4967946415319141, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 0.50)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 1.60305298661
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 1.6030529866136418, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 1.60)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 1.27691927913
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 1.2769192791260928, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent drove right instead of left. (rewarded 1.28)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 1.84863528363
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.8486352836340818, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 1.85)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: -5.99004685174
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 19, 't': 11, 'action': None, 'reward': -5.990046851736325, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.99)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 0.218806729895
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 0.21880672989498995, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent drove right instead of forward. (rewarded 0.22)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 2.65588628439
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 13, 'action': None, 'reward': 2.655886284392328, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.66)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 2.00183693308
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 16, 't': 14, 'action': None, 'reward': 2.001836933084441, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 2.00)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 2.19422686221
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 15, 'action': None, 'reward': 2.1942268622131147, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.19)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: -19.186483878
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 14, 't': 16, 'action': 'right', 'reward': -19.186483878039237, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.19)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: 2.54043012668
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 13, 't': 17, 'action': 'left', 'reward': 2.5404301266809832, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 2.54)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.25196525664
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.251965256638216, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.25)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 0.966159466614
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 0.966159466613907, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.97)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: left, reward: 1.39530835228
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 20, 'action': 'left', 'reward': 1.3953083522805483, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.40)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 3
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (2, 5), deadline = 30
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.90982044288
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.9098204428825176, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.91)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.48897760911
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.488977609107577, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.49)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.04627874527
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.0462787452675208, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.05)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: right, reward: 1.4222730275
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 1.4222730275004682, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 1.42)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 2.37142861151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.3714286115067438, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.37)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 2.26982811045
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.2698281104461837, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.27)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: 2.87479871215
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 2.8747987121463403, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.87)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: -4.99296997775
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 23, 't': 7, 'action': None, 'reward': -4.992969977751834, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.99)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: -4.45116718598
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'forward'), 'deadline': 22, 't': 8, 'action': None, 'reward': -4.45116718597503, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.45)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: left, reward: 1.56430303011
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 1.5643030301073844, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent drove left instead of forward. (rewarded 1.56)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 2.60568371881
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 2.6056837188083346, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 2.61)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: right, reward: 0.90927247286
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 0.9092724728599674, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent drove right instead of forward. (rewarded 0.91)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 1.46465522275
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 1.4646552227543816, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent drove right instead of left. (rewarded 1.46)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 1.8094344614
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 17, 't': 13, 'action': None, 'reward': 1.8094344613992541, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.81)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 1.56882841873
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.5688284187293922, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.57)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: -20.231518395
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': -20.2315183950215, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.23)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 2.70448338201
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 2.7044833820091383, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.70)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 2.16766757763
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': 2.1676675776327836, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.17)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 0.0866250514317
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 12, 't': 18, 'action': 'forward', 'reward': 0.08662505143172705, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 0.09)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: -10.4049140192
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': -10.40491401924339, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent attempted driving forward through a red light. (rewarded -10.40)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 1.01953134079
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 1.0195313407883253, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 1.02)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 1.85733256728
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 1.857332567283906, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.86)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: left, reward: -20.5087576031
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'forward'), 'deadline': 8, 't': 22, 'action': 'left', 'reward': -20.50875760305474, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.51)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: -20.3922978942
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'left'), 'deadline': 7, 't': 23, 'action': 'right', 'reward': -20.39229789418493, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.39)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 1.05323555313
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 1.0532355531302409, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.05)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: forward, reward: 0.694765844732
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 5, 't': 25, 'action': 'forward', 'reward': 0.6947658447321332, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 0.69)
13% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 4
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (6, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 2.51899012199
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.5189901219877444, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', 'left')
Agent followed the waypoint forward. (rewarded 2.52)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 1.94468276958
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'left'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.9446827695763453, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'left')
Agent followed the waypoint forward. (rewarded 1.94)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: left, reward: 0.421160054481
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 0.4211600544807744, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 0.42)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: forward, reward: 0.711103459568
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 0.7111034595677194, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove forward instead of right. (rewarded 0.71)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 2.63516462259
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.6351646225919385, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.64)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: -19.8191229016
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'right'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': -19.819122901563237, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'right')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.82)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 1.75461363554
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.7546136355420814, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.75)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: forward, reward: 1.76756083694
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.7675608369366087, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent followed the waypoint forward. (rewarded 1.77)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 1.11574781044
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'right'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.1157478104372123, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'right')
Agent drove right instead of forward. (rewarded 1.12)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 0.863018988021
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.8630189880210521, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.86)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: -0.0121822047261
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': -0.012182204726115642, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded -0.01)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: forward, reward: 1.42385795026
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.423857950263305, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.42)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: left, reward: -9.80642593334
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'right', None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -9.806425933337197, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', None)
Agent attempted driving left through a red light. (rewarded -9.81)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: forward, reward: 1.15458239915
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.1545823991475488, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.15)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.62641088321
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.626410883211602, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.63)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 0.811769190374
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.8117691903738937, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.81)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: right, reward: 2.15429951314
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 2.154299513138633, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.15)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 0.86419561875
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.8641956187496505, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 0.86)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 0.598169450739
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.5981694507394464, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.60)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 0.360743834776
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.36074383477582994, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove right instead of forward. (rewarded 0.36)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 5
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (6, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: forward, reward: 1.7910817134
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.7910817134008432, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove forward instead of right. (rewarded 1.79)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: forward, reward: 0.573944107153
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 0.5739441071531075, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded 0.57)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 2.26147227622
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 2.261472276215751, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.26)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: left, reward: 1.10927520515
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 1.109275205147641, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 1.11)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 2.47077604142
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.470776041424429, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.47)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 1.73393578378
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.7339357837846168, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.73)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 0.652109730149
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.6521097301491529, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.65)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: 2.6040355038
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'left'), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 2.60403550379767, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'left')
Agent followed the waypoint left. (rewarded 2.60)
68% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 6
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (2, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: right, reward: 1.33825813135
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'right'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.3382581313468527, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'right')
Agent followed the waypoint right. (rewarded 1.34)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 1.81381955896
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.8138195589566082, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.81)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 1.67759086974
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.677590869735904, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.68)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: left, reward: -40.9999366345
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -40.99993663453393, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -41.00)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: right, reward: 2.6670980663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.6670980662995682, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.67)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 2.6156050981
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.6156050981042887, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.62)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 0.543786836827
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.5437868368265502, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.54)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: left, reward: 2.09953152796
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 2.099531527962248, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.10)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: left, reward: 1.96794619239
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.9679461923899775, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.97)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 2.26293868044
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.2629386804369958, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.26)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 2.50231398252
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.502313982520163, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.50)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: left, reward: 0.775908288013
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 0.7759082880134716, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.78)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 2.30284561071
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 2.302845610708254, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.30)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 2.23112513707
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 2.2311251370724676, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 2.23)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 7
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (2, 7), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: 1.83972891388
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.8397289138805282, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.84)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 2.77678979644
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.7767897964449224, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.78)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: -9.65711232512
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', 'left'), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': -9.657112325121698, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', 'left')
Agent attempted driving forward through a red light. (rewarded -9.66)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 0.938158143516
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 0.938158143515603, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.94)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: forward, reward: 0.804820886782
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 0.8048208867819916, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.80)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 2.56935556637
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.5693555663673706, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.57)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: forward, reward: -10.5887669602
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -10.58876696017072, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -10.59)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 1.75080448402
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.7508044840214474, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.75)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: -5.97135963682
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 17, 't': 8, 'action': None, 'reward': -5.971359636823593, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.97)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: forward, reward: 1.23056304538
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.2305630453772847, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove forward instead of left. (rewarded 1.23)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: forward, reward: 0.4728099704
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 0.47280997040012995, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.47)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 2.78802086511
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.788020865114107, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.79)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: forward, reward: -10.421881823
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': -10.421881822987034, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -10.42)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: left, reward: 2.64777136522
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 2.6477713652174297, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.65)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: left, reward: 0.833704031582
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 0.8337040315821388, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.83)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: forward, reward: 0.224227570414
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 0.22422757041420316, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.22)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: left, reward: 0.447703213357
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 0.4477032133574813, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.45)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: -0.410669453173
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': -0.41066945317267134, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded -0.41)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: left, reward: 0.77776556245
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 7, 't': 18, 'action': 'left', 'reward': 0.7777655624496618, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.78)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: forward, reward: 1.03663467358
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 1.036634673582137, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent drove forward instead of right. (rewarded 1.04)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: forward, reward: 0.181259238559
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 5, 't': 20, 'action': 'forward', 'reward': 0.18125923855936166, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded 0.18)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: right, reward: 0.637104658239
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 0.6371046582386859, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.64)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: right, reward: 0.900079223529
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.900079223528715, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded 0.90)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 0.520287888141
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 2, 't': 23, 'action': None, 'reward': 0.5202878881409074, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.52)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 0.214885224362
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'right'), 'deadline': 1, 't': 24, 'action': 'right', 'reward': 0.2148852243618462, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'right', 'left', 'right')
Agent drove right instead of left. (rewarded 0.21)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 8
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (5, 7), deadline = 30
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: left, reward: -39.6698572364
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', 'left'), 'deadline': 30, 't': 0, 'action': 'left', 'reward': -39.66985723644549, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.67)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 1.18886665127
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.1888666512670971, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent drove right instead of left. (rewarded 1.19)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 1.61630195193
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 1.6163019519347581, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent drove right instead of forward. (rewarded 1.62)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 1.3832788567
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 1.383278856698663, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 1.38)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.39737578023
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.397375780227526, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.40)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 1.30060407429
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.3006040742888918, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 1.30)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: 1.60764264321
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 1.6076426432085955, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.61)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 1.19510146684
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.1951014668359328, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 1.20)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 0.0563936452032
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 0.05639364520323242, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.06)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: 0.925466696923
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 0.9254666969230336, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 0.93)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 0.122645716685
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 0.12264571668520707, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent drove right instead of forward. (rewarded 0.12)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: left, reward: 2.2710714738
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 2.2710714737963986, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.27)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 2.38435759154
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.3843575915414847, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.38)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: -4.95357289175
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 17, 't': 13, 'action': None, 'reward': -4.953572891752239, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.95)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: left, reward: 1.7050932317
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 16, 't': 14, 'action': 'left', 'reward': 1.7050932317007397, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove left instead of forward. (rewarded 1.71)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: 0.360195242335
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 15, 't': 15, 'action': 'left', 'reward': 0.3601952423350304, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent drove left instead of right. (rewarded 0.36)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 2.28536153166
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 14, 't': 16, 'action': None, 'reward': 2.285361531655072, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.29)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 0.831515158372
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'right'), 'deadline': 13, 't': 17, 'action': None, 'reward': 0.831515158371801, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'right')
Agent properly idled at a red light. (rewarded 0.83)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 1.59068391941
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.59068391941082, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.59)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: 1.16439043613
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 11, 't': 19, 'action': 'left', 'reward': 1.1643904361293798, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove left instead of right. (rewarded 1.16)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: forward, reward: 0.657994530944
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 10, 't': 20, 'action': 'forward', 'reward': 0.6579945309439643, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 0.66)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: -0.382910746278
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': -0.382910746277627, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded -0.38)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: 1.82072624731
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 8, 't': 22, 'action': 'left', 'reward': 1.8207262473103605, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.82)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 0.625282112216
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 7, 't': 23, 'action': None, 'reward': 0.6252821122156373, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.63)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 0.94416743101
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 0.9441674310096804, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.94)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.39158850473
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 5, 't': 25, 'action': None, 'reward': 1.3915885047279, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.39)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: 0.304914406904
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 26, 'action': 'left', 'reward': 0.3049144069043124, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.30)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 0.458870624011
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 27, 'action': 'right', 'reward': 0.4588706240109526, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.46)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 1.641946771
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 28, 'action': None, 'reward': 1.6419467709954207, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.64)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: -0.366535050953
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 1, 't': 29, 'action': 'right', 'reward': -0.36653505095253913, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'right', None, None)
Agent drove right instead of left. (rewarded -0.37)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 9
\-------------------------
Environment.reset(): Trial set up with start = (7, 6), destination = (4, 3), deadline = 30
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 2.69394066273
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'right'), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': 2.6939406627288927, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'right')
Agent followed the waypoint forward. (rewarded 2.69)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: left, reward: 1.17358455928
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 29, 't': 1, 'action': 'left', 'reward': 1.1735845592767626, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 1.17)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 1.17647259544
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'forward'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 1.1764725954418032, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 1.18)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.78754313534
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.7875431353428266, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.79)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 2.21015064885
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 2.2101506488491816, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.21)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 1.35922445778
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.3592244577838717, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.36)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 1.55198043635
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.551980436352817, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.55)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: left, reward: 1.08413368828
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 1.0841336882812491, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.08)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: left, reward: -10.4815466853
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 22, 't': 8, 'action': 'left', 'reward': -10.481546685255843, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent attempted driving left through a red light. (rewarded -10.48)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 2.32334967876
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 9, 'action': None, 'reward': 2.3233496787554175, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.32)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: forward, reward: 1.73370531114
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 1.7337053111421272, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.73)
63% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 10
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (1, 4), deadline = 30
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: 2.25242147591
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', 'left'), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': 2.252421475907218, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', 'left')
Agent followed the waypoint forward. (rewarded 2.25)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 1.32705477987
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.327054779873051, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.33)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: -9.18974395645
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'left'), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': -9.18974395645185, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -9.19)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: right, reward: 0.613378781146
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 0.6133787811455736, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent drove right instead of left. (rewarded 0.61)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.07652656446
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.076526564464454, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.08)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.70980874049
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.709808740491222, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.71)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: left, reward: 2.37350703735
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 2.3735070373533844, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.37)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.38770346921
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', 'right'), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.387703469205673, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', 'right')
Agent properly idled at a red light. (rewarded 1.39)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.60325768685
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.6032576868516444, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.60)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: forward, reward: 1.66953064985
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.66953064984648, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.67)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 1.32906587237
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.3290658723708837, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.33)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 2.8060828552
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.8060828551967383, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.81)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 1.9241304919
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.9241304918973505, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.92)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: forward, reward: 1.6386546203
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.6386546202966181, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.64)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: left, reward: 1.85965475981
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 16, 't': 14, 'action': 'left', 'reward': 1.8596547598053736, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 1.86)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 2.54932685627
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': 2.5493268562742974, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.55)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: -40.79495701
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': -40.7949570100065, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.79)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.74878494333
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.7487849433349394, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.75)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.11322992955
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.113229929552109, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.11)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 0.922551474854
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 11, 't': 19, 'action': None, 'reward': 0.9225514748543153, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.92)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 0.146929334619
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 0.14692933461904523, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.15)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 0.595431186049
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 9, 't': 21, 'action': None, 'reward': 0.5954311860485033, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 0.60)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: 0.30454946581
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': 0.3045494658096748, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.30)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: 1.85746822916
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 7, 't': 23, 'action': 'left', 'reward': 1.8574682291574725, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent followed the waypoint left. (rewarded 1.86)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 1.04533260662
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 1.0453326066222617, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.05)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 1.05878559362
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 5, 't': 25, 'action': None, 'reward': 1.0587855936218462, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.06)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: -0.224533410165
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 4, 't': 26, 'action': 'left', 'reward': -0.2245334101647556, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove left instead of forward. (rewarded -0.22)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.14650098638
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 3, 't': 27, 'action': None, 'reward': 1.1465009863752067, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.15)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 1.20489969779
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 28, 'action': 'right', 'reward': 1.2048996977856887, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.20)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: forward, reward: -0.198943992055
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 1, 't': 29, 'action': 'forward', 'reward': -0.19894399205473756, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded -0.20)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
Simulation ended. . .
<matplotlib.figure.Figure at 0xde47940>
In [21]:
# self.epsilon = self.epsilon - 0.05
# Load the 'sim_improved-learning' file from the improved Q-Learning simulation
vs.plot_trials('sim_improved-learning.csv')
In [22]:
%run smartcab\agent.py
#可以观察到可视化程序的各部分的功能
/-------------------------
| Training trial 1
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (1, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0100; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0100; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: 2.04738844858
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.0473884485844387, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.05)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: -5.79018301027
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', None), 'deadline': 19, 't': 1, 'action': None, 'reward': -5.79018301026788, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.79)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 1.10803794782
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.1080379478227198, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 1.11)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: left, reward: -39.9441233493
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -39.9441233492706, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.94)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 2.78445605581
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.7844560558129428, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.78)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: 2.34358626878
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.3435862687760274, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 2.34)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 2.6725098848
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.672509884802854, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.67)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: left, reward: -10.1564197436
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': -10.15641974355043, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.16)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: -0.138620390437
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': -0.13862039043733299, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent drove right instead of forward. (rewarded -0.14)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: -20.8287788627
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'right'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': -20.82877886271031, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'right')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.83)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 0.758892736034
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.75889273603401, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.76)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: forward, reward: -10.6893986818
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': -10.689398681811939, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -10.69)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: -39.7402854025
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -39.74028540254514, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.74)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: -5.42881079449
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': -5.428810794489869, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.43)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: -4.3187624258
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': -4.318762425804994, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.32)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: forward, reward: -10.4700158419
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -10.470015841861997, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.47)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: -19.434416796
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': -19.434416795984227, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.43)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: 1.10840883885
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 1.1084088388532969, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove left instead of right. (rewarded 1.11)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 2.19098288163
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 2.1909828816259767, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.19)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: -10.1084978476
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 1, 't': 19, 'action': 'left', 'reward': -10.108497847622711, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent attempted driving left through a red light. (rewarded -10.11)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 2
\-------------------------
Environment.reset(): Trial set up with start = (3, 2), destination = (8, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0001; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 1.09844783947
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.0984478394709505, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.10)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 1.16019299815
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.1601929981464905, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.16)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: -19.7132217187
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': -19.713221718663082, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.71)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: -4.86728016185
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 22, 't': 3, 'action': None, 'reward': -4.867280161853113, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.87)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: left, reward: 0.843537910685
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 0.8435379106851127, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 0.84)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 2.59454845284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 2.5945484528387395, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.59)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: left, reward: 0.735137873936
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 0.7351378739361635, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.74)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: None, reward: 2.68344437438
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.683444374376978, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.68)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: right, reward: 2.73636272096
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 2.736362720959846, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.74)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: left, reward: 1.61263283597
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.6126328359748232, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent followed the waypoint left. (rewarded 1.61)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 3
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (1, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: -10.1529088316
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -10.152908831633594, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -10.15)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: left, reward: -10.8442589459
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -10.844258945883784, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent attempted driving left through a red light. (rewarded -10.84)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 1.75757059774
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.7575705977360656, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.76)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 1.09158244661
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.0915824466136443, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.09)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 1.50624435195
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.506244351950674, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.51)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: right, reward: 0.805246374933
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.8052463749332444, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'left')
Agent drove right instead of left. (rewarded 0.81)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: left, reward: 1.27207205174
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.272072051736432, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.27)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: 1.12262290271
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.1226229027069239, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 1.12)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: forward, reward: 0.285487278303
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 0.2854872783027532, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 0.29)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 1.72220571883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.7222057188254212, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.72)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: right, reward: 0.698065232653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.6980652326525284, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.70)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: 0.168104382604
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 0.16810438260367855, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.17)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: left, reward: -0.170951425044
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -0.17095142504413396, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded -0.17)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: 0.927224785474
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.9272247854743763, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.93)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: -19.2456617091
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -19.245661709073694, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.25)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: -9.58918805446
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -9.589188054458672, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.59)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 2.02541478362
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 2.0254147836196736, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.03)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 1.59384469968
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 1.593844699678519, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.59)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: left, reward: -40.2968427279
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'right'), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -40.29684272790461, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.30)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: left, reward: -39.5338991747
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'right'), 'deadline': 1, 't': 19, 'action': 'left', 'reward': -39.53389917470148, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', 'forward', None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.53)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 4
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (1, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: left, reward: 1.80485780469
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.8048578046868942, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'left')
Agent drove left instead of forward. (rewarded 1.80)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: left, reward: 0.985030364446
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 0.9850303644462731, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent drove left instead of right. (rewarded 0.99)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 1.60014403987
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.600144039869556, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.60)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: 1.3698772583
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.3698772583039407, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 1.37)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 1.39807827396
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.3980782739563444, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.40)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 0.306639118709
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 0.306639118708524, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.31)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: forward, reward: 0.251988785236
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 0.25198878523624657, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded 0.25)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: left, reward: 1.40851488838
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.4085148883796061, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.41)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 1.03877114541
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.0387711454075839, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.04)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 2.02974052449
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.0297405244864706, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.03)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 1.35375166584
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.3537516658377564, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent drove right instead of forward. (rewarded 1.35)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: left, reward: -10.4105840138
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': -10.41058401377126, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent attempted driving left through a red light. (rewarded -10.41)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 1.44001646889
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.4400164688918722, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.44)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: 2.26081557667
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 2.2608155766724627, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.26)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.41437116614
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.4143711661448666, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.41)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: -40.594194507
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -40.5941945069667, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.59)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 1.20593169443
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.2059316944270018, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent followed the waypoint forward. (rewarded 1.21)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.61583411861
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.6158341186131808, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 1.62)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.10484304627
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.104843046273046, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.10)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: -39.9327154222
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -39.93271542220476, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.93)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 5
\-------------------------
Environment.reset(): Trial set up with start = (7, 4), destination = (3, 7), deadline = 35
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 1.42920831365
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'forward'), 'deadline': 35, 't': 0, 'action': 'right', 'reward': 1.42920831364861, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 1.43)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: left, reward: -10.5174972266
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', 'right', None), 'deadline': 34, 't': 1, 'action': 'left', 'reward': -10.517497226625007, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', None)
Agent attempted driving left through a red light. (rewarded -10.52)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: right, reward: 1.63681960498
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 33, 't': 2, 'action': 'right', 'reward': 1.6368196049761887, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.64)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: right, reward: 0.714421550751
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 32, 't': 3, 'action': 'right', 'reward': 0.7144215507511789, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded 0.71)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: right, reward: 1.38950809583
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 31, 't': 4, 'action': 'right', 'reward': 1.3895080958321802, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.39)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: left, reward: 1.20018147299
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 30, 't': 5, 'action': 'left', 'reward': 1.2001814729865197, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove left instead of right. (rewarded 1.20)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.40836198046
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 29, 't': 6, 'action': None, 'reward': 1.4083619804601117, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.41)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: 1.22879428474
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 28, 't': 7, 'action': 'left', 'reward': 1.228794284743588, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 1.23)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: -10.2138559269
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 27, 't': 8, 'action': 'left', 'reward': -10.213855926914146, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent attempted driving left through a red light. (rewarded -10.21)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.56289199627
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 26, 't': 9, 'action': None, 'reward': 1.5628919962731016, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.56)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.48464997966
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 25, 't': 10, 'action': None, 'reward': 2.4846499796612793, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.48)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: -0.0694867351696
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 24, 't': 11, 'action': 'right', 'reward': -0.06948673516957482, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent drove right instead of forward. (rewarded -0.07)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 1.42652346789
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 23, 't': 12, 'action': 'right', 'reward': 1.4265234678942411, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 1.43)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: -0.0871557138019
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 22, 't': 13, 'action': 'forward', 'reward': -0.08715571380189802, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded -0.09)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 1.34078166517
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 21, 't': 14, 'action': 'right', 'reward': 1.3407816651666735, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded 1.34)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: left, reward: 0.865605122883
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 15, 'action': 'left', 'reward': 0.865605122883085, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.87)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: -0.0263897523697
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 19, 't': 16, 'action': 'right', 'reward': -0.02638975236973351, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded -0.03)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: left, reward: -19.1578639871
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 18, 't': 17, 'action': 'left', 'reward': -19.15786398712045, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.16)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: left, reward: 1.85990785899
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 17, 't': 18, 'action': 'left', 'reward': 1.8599078589901985, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent followed the waypoint left. (rewarded 1.86)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: 2.42705378096
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 19, 'action': 'forward', 'reward': 2.4270537809604744, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.43)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: right, reward: 1.60907329746
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 20, 'action': 'right', 'reward': 1.6090732974610882, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.61)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: forward, reward: 1.89204347963
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 21, 'action': 'forward', 'reward': 1.8920434796330847, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.89)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: forward, reward: 2.09567991439
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 22, 'action': 'forward', 'reward': 2.095679914386012, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.10)
34% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 6
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (4, 6), deadline = 30
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: right, reward: 1.02949911082
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.029499110822289, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 1.03)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: -5.78829093594
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'right', 'left', None), 'deadline': 29, 't': 1, 'action': None, 'reward': -5.7882909359394725, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.79)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: forward, reward: 1.29831889237
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 1.2983188923733833, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded 1.30)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: forward, reward: 0.292866904678
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 0.2928669046782809, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 0.29)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: left, reward: -10.3654226894
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 26, 't': 4, 'action': 'left', 'reward': -10.36542268940165, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent attempted driving left through a red light. (rewarded -10.37)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 1.73634380666
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.736343806657965, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.74)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.73120148296
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.7312014829604365, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.73)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: left, reward: 0.942119436652
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 0.9421194366516285, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.94)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: forward, reward: 0.126605650282
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': 0.12660565028193604, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.13)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: -39.155714621
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': -39.15571462095014, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.16)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: -9.17835927117
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 20, 't': 10, 'action': 'left', 'reward': -9.178359271172686, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -9.18)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: forward, reward: -10.6337680846
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': -10.63376808464407, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.63)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: 2.80985702356
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.809857023557843, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.81)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: 1.25228358421
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 1.2522835842106055, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 1.25)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: -39.0308758857
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', None), 'deadline': 16, 't': 14, 'action': 'left', 'reward': -39.030875885704944, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.03)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: -9.00152647315
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': -9.001526473154492, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent attempted driving forward through a red light. (rewarded -9.00)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.10510523096
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 16, 'action': None, 'reward': 1.1051052309587523, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.11)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: 1.84148962996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': 1.8414896299623016, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.84)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 1.60077973393
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.6007797339259981, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.60)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 1.48008992584
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 1.4800899258436564, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.48)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 0.971917594272
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 0.9719175942724061, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.97)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: -4.35583370767
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 9, 't': 21, 'action': None, 'reward': -4.355833707671276, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.36)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: left, reward: 0.475775017216
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 8, 't': 22, 'action': 'left', 'reward': 0.47577501721609927, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent drove left instead of right. (rewarded 0.48)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 1.92835257537
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'right'), 'deadline': 7, 't': 23, 'action': None, 'reward': 1.928352575366447, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.93)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 0.179973006868
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 0.1799730068680685, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.18)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: left, reward: 0.256564894335
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 5, 't': 25, 'action': 'left', 'reward': 0.25656489433497254, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.26)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 2.04623498495
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'left'), 'deadline': 4, 't': 26, 'action': 'right', 'reward': 2.0462349849450403, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 2.05)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: forward, reward: 1.87322091723
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 27, 'action': 'forward', 'reward': 1.8732209172291607, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.87)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: -0.600030917281
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 2, 't': 28, 'action': 'right', 'reward': -0.6000309172814678, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded -0.60)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 0.417753181284
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 1, 't': 29, 'action': None, 'reward': 0.41775318128372385, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.42)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 7
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (3, 7), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: left, reward: -40.5072987081
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 25, 't': 0, 'action': 'left', 'reward': -40.507298708079105, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.51)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 2.21805503306
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.218055033057694, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.22)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 1.99036799849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.9903679984866802, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.99)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 1.07836090776
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.0783609077645018, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.08)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 0.295888154096
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 0.29588815409567115, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.30)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 1.45326645667
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.4532664566653675, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove right instead of forward. (rewarded 1.45)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: left, reward: -39.3034914952
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': -39.303491495195345, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.30)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 1.19613101469
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.1961310146943038, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.20)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 1.87260373142
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.8726037314159845, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.87)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 2.73441697204
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.734416972038309, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.73)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 0.918130013055
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 0.91813001305532, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.92)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: forward, reward: 0.939118704923
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 0.9391187049227644, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.94)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 1.34276293798
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.3427629379825596, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.34)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 1.38926374743
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.3892637474259208, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 1.39)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: left, reward: 0.895747584829
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 0.8957475848292562, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent followed the waypoint left. (rewarded 0.90)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 1.06552961374
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.065529613739852, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.07)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: left, reward: 2.57196281531
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 2.571962815310938, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent followed the waypoint left. (rewarded 2.57)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 1.95349319298
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'right'), 'deadline': 8, 't': 17, 'action': None, 'reward': 1.9534931929750663, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.95)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 1.06228034744
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.0622803474419285, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.06)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: -0.150424089774
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': -0.15042408977431465, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent drove right instead of forward. (rewarded -0.15)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: 1.38522674517
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 20, 'action': 'left', 'reward': 1.3852267451710951, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.39)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: -9.01186299863
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': -9.011862998633775, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.01)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: 1.32048727655
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': 1.320487276546079, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.32)
8% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 8
\-------------------------
Environment.reset(): Trial set up with start = (3, 2), destination = (8, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 0.385192381034
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 0.3851923810340172, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.39)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 1.74666805306
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'right'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.746668053057477, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'right')
Agent followed the waypoint right. (rewarded 1.75)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.18109251529
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.1810925152941183, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.18)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: left, reward: 0.888650988008
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 0.8886509880081062, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.89)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 1.07068098718
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.07068098717598, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.07)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: left, reward: -40.1010829322
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': -40.101082932177405, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.10)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.85142940586
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.8514294058568836, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.85)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 1.06214919007
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.0621491900658067, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.06)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 1.25867819607
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.258678196074722, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.26)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: left, reward: -10.0396893231
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': -10.039689323075091, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.04)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: -0.109785604086
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': -0.10978560408575999, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded -0.11)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 2.22419968558
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 2.2241996855845794, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.22)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 0.0970256998245
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 0.09702569982445786, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded 0.10)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: left, reward: 2.2601197407
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 2.2601197407013593, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.26)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: left, reward: 0.653901567905
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'right', None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 0.6539015679048007, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'right', None)
Agent drove left instead of right. (rewarded 0.65)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.1247331612
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.1247331611963345, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.12)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.64444452799
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.64444452798995, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.64)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: -5.79806175134
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 8, 't': 17, 'action': None, 'reward': -5.798061751342667, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.80)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: left, reward: -0.175452890383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 7, 't': 18, 'action': 'left', 'reward': -0.1754528903829975, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove left instead of forward. (rewarded -0.18)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: forward, reward: -0.515802924087
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', None), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': -0.5158029240866874, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', None)
Agent drove forward instead of right. (rewarded -0.52)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: 0.317634086772
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 5, 't': 20, 'action': 'forward', 'reward': 0.31763408677175664, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove forward instead of right. (rewarded 0.32)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 1.24237406562
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 1.2423740656235274, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.24)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: -4.28614668888
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 3, 't': 22, 'action': None, 'reward': -4.286146688884324, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.29)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: -5.94477538874
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 2, 't': 23, 'action': None, 'reward': -5.944775388737278, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.94)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: left, reward: 0.490328708328
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 1, 't': 24, 'action': 'left', 'reward': 0.49032870832790776, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent drove left instead of right. (rewarded 0.49)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 9
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (7, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 1.51546507842
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'left'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.5154650784221733, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'left')
Agent drove forward instead of right. (rewarded 1.52)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 0.401618208881
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 0.4016182088809894, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent drove forward instead of right. (rewarded 0.40)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 2.69151863184
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.691518631841579, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.69)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 2.83511178324
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 2.8351117832432333, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.84)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 1.26984516316
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.2698451631560033, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.27)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.09846990007
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.098469900074682, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.10)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: -9.98593067004
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -9.985930670041554, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', None)
Agent attempted driving forward through a red light. (rewarded -9.99)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: -9.97015630928
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': -9.97015630928423, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -9.97)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 2.13946174276
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.1394617427595923, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.14)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 1.03295864224
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.0329586422430117, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.03)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: 2.78371064776
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 2.7837106477623736, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.78)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.52368144015
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.5236814401469045, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.52)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 0.404834823479
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 0.4048348234793915, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.40)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 0.736452054448
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 0.7364520544481452, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove right instead of forward. (rewarded 0.74)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: forward, reward: -9.8663705625
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': -9.866370562497439, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.87)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 1.00115634028
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.0011563402764627, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.00)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.47730390839
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.4773039083933763, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.48)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: left, reward: 1.9740076283
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 1.974007628304789, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.97)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 0.711432936507
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'right'), 'deadline': 7, 't': 18, 'action': None, 'reward': 0.7114329365069283, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 0.71)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 0.261774293381
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 0.26177429338063585, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.26)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 2.26108358217
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 5, 't': 20, 'action': None, 'reward': 2.2610835821659445, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.26)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: left, reward: 0.567699198302
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 0.5676991983024056, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.57)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: None, reward: -4.21394732127
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 3, 't': 22, 'action': None, 'reward': -4.213947321272375, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.21)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: 1.44355146
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 23, 'action': 'left', 'reward': 1.443551459997832, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.44)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: -0.017252422285
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 1, 't': 24, 'action': 'right', 'reward': -0.017252422285037428, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove right instead of forward. (rewarded -0.02)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 10
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (6, 7), deadline = 30
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: 1.2394773528
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.2394773528026248, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.24)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 1.1689435805
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 1.1689435804951118, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.17)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 0.24292963029
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 0.24292963029006998, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.24)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: forward, reward: 1.40798792309
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 1.4079879230934957, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 1.41)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: forward, reward: 1.03509281886
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 1.0350928188619082, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.04)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: left, reward: 1.33638012074
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 1.336380120735082, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.34)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: left, reward: 1.0257731375
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 1.02577313749611, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 1.03)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 2.04874390492
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 2.048743904923184, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.05)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: left, reward: -9.33095513669
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 22, 't': 8, 'action': 'left', 'reward': -9.330955136686066, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent attempted driving left through a red light. (rewarded -9.33)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 1.30508524254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'left'), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.3050852425358832, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.31)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 1.4278831396
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', 'left'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.4278831395981575, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', 'left')
Agent followed the waypoint right. (rewarded 1.43)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: forward, reward: -40.3520401215
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': -40.352040121508495, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.35)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 1.41366952135
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.4136695213506887, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.41)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 2.03688411777
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 13, 'action': None, 'reward': 2.0368841177677544, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.04)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 1.40811243499
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.4081124349872916, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.41)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: left, reward: -39.5489097565
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 15, 't': 15, 'action': 'left', 'reward': -39.54890975652309, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.55)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: 0.425733159739
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 0.4257331597390013, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.43)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: 0.719872483699
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 0.7198724836992323, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.72)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 1.1034442382
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.1034442381977916, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.10)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 1.70519505847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 1.705195058465741, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.71)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 1.8138775116
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 1.8138775115960726, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.81)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 1.82053359272
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 9, 't': 21, 'action': None, 'reward': 1.8205335927240056, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.82)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 1.79898080849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 22, 'action': None, 'reward': 1.798980808486242, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.80)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 2.16645873019
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 7, 't': 23, 'action': None, 'reward': 2.1664587301855756, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.17)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: 0.0176636031621
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 0.017663603162142882, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.02)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.4218411233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 5, 't': 25, 'action': None, 'reward': 1.4218411232990955, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.42)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: 0.78645978664
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 4, 't': 26, 'action': 'right', 'reward': 0.7864597866396432, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.79)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: forward, reward: -40.03933028
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 3, 't': 27, 'action': 'forward', 'reward': -40.03933028000041, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.04)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 0.261599797606
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 2, 't': 28, 'action': 'right', 'reward': 0.26159979760579066, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.26)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: forward, reward: 0.994879794327
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 1, 't': 29, 'action': 'forward', 'reward': 0.9948797943272385, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.99)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 11
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (5, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 2.43114488395
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.4311448839520375, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent followed the waypoint right. (rewarded 2.43)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 1.4732099046
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.473209904598158, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.47)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: right, reward: 0.0442918086436
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.04429180864359028, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove right instead of forward. (rewarded 0.04)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: 1.32654572332
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.3265457233238669, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 1.33)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 1.42063002267
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.4206300226719075, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.42)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: right, reward: 1.1788224294
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.1788224293954328, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.18)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 0.402263614485
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.4022636144850118, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.40)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: None, reward: 2.65428336755
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.6542833675532576, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.65)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 0.0775705802229
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.0775705802229224, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.08)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.61487932678
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.614879326775629, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.61)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 0.799079223211
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 0.7990792232114743, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 0.80)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.20782380239
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.2078238023912844, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.21)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: left, reward: -10.5752482778
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'right', 'left'), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -10.575248277770156, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', 'left')
Agent attempted driving left through a red light. (rewarded -10.58)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 0.685001365925
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.6850013659251418, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', 'left')
Agent drove forward instead of left. (rewarded 0.69)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: 1.2567827108
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 1.256782710795631, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.26)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 0.916865291124
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 0.9168652911242283, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.92)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 1.17005041794
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.1700504179411015, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.17)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 0.353505728201
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.3535057282011238, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 0.35)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 0.626516532613
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.6265165326130588, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.63)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 0.27011057876
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 0.270110578760256, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 0.27)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 12
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (5, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 1.75749045477
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'right'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.757490454768059, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'right')
Agent drove right instead of left. (rewarded 1.76)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 1.02210084322
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.022100843215501, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.02)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 2.11717910774
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.117179107737429, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.12)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 2.23564273182
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.2356427318171948, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.24)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 2.41376225608
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.4137622560782654, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.41)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 0.660126442051
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.6601264420514273, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.66)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: 1.09825573185
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.0982557318504207, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.10)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.67221278067
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.672212780671668, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.67)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 1.03204623409
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.0320462340932974, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.03)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 1.25703296677
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.2570329667657847, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.26)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 1.75133673903
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.7513367390320624, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.75)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: -4.12771998955
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 14, 't': 11, 'action': None, 'reward': -4.127719989550621, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.13)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: forward, reward: 1.51906495602
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 1.5190649560222396, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 1.52)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 0.663870760095
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 12, 't': 13, 'action': None, 'reward': 0.6638707600946312, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.66)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: left, reward: -20.3255345899
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -20.32553458985951, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.33)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: 2.10427289157
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 2.1042728915675863, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent followed the waypoint left. (rewarded 2.10)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.10442595676
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 1.1044259567552084, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.10)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: 1.25601665326
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 1.2560166532642545, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded 1.26)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: 0.0415852494597
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 0.04158524945972286, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.04)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 0.683360464162
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'right', None), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 0.6833604641624134, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', None)
Agent followed the waypoint right. (rewarded 0.68)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 0.782637939329
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 0.782637939328634, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.78)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 0.825073488113
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 21, 'action': None, 'reward': 0.8250734881130573, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.83)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: left, reward: -9.18346465723
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 3, 't': 22, 'action': 'left', 'reward': -9.183464657230884, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent attempted driving left through a red light. (rewarded -9.18)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: 1.08869300943
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 2, 't': 23, 'action': 'forward', 'reward': 1.088693009432754, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.09)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: -0.649612997856
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'right'), 'deadline': 1, 't': 24, 'action': 'right', 'reward': -0.6496129978563441, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, 'left', 'right')
Agent drove right instead of forward. (rewarded -0.65)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 13
\-------------------------
Environment.reset(): Trial set up with start = (2, 7), destination = (6, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 2.24094521612
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.2409452161218164, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.24)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 1.70231505177
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.702315051773217, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.70)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 1.5518290689
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.5518290689014131, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.55)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: left, reward: 0.851710611697
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 0.8517106116973108, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove left instead of right. (rewarded 0.85)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 2.55794768353
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.557947683531992, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.56)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: 2.02345921471
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.0234592147127275, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.02)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: 1.70016932951
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.7001693295066858, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.70)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: 1.67980844914
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'forward'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.6798084491359702, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'forward')
Agent drove right instead of forward. (rewarded 1.68)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 0.958603165093
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.958603165092809, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove right instead of left. (rewarded 0.96)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 2.02116406837
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 2.021164068365093, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.02)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: left, reward: -39.9182521542
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': -39.91825215415066, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.92)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: 1.66887789783
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.6688778978324272, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.67)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: 0.465658599448
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 0.465658599447637, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.47)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 1.06675066175
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.066750661747524, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.07)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 2.59089033513
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 2.5908903351321895, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent followed the waypoint right. (rewarded 2.59)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: 2.53686084377
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', 'right'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 2.5368608437729416, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', 'right')
Agent followed the waypoint right. (rewarded 2.54)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 2.53327815804
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 2.533278158039328, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.53)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.80886490781
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 17, 'action': None, 'reward': 1.80886490780977, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.81)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.02290167329
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 2.0229016732880254, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.02)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 0.868367431304
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 6, 't': 19, 'action': None, 'reward': 0.8683674313039913, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.87)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: left, reward: -39.8018917824
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', 'left'), 'deadline': 5, 't': 20, 'action': 'left', 'reward': -39.801891782350275, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.80)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: forward, reward: 0.153040303746
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 0.15304030374605826, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.15)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: 0.798294717098
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.7982947170984276, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 0.80)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: -5.20604448076
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 2, 't': 23, 'action': None, 'reward': -5.206044480758664, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.21)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: forward, reward: 0.0679917706305
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': 0.06799177063049178, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded 0.07)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 14
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (4, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: forward, reward: 1.58908978411
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.5890897841088725, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded 1.59)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: left, reward: -40.0440759072
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -40.04407590722956, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.04)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 2.82304515617
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.8230451561706635, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.82)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 0.343653376138
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 0.3436533761380439, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.34)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.64603311854
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.646033118537301, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.65)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.69676652155
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.696766521545441, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.70)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: left, reward: 0.0171360020418
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 0.017136002041803433, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent drove left instead of forward. (rewarded 0.02)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: right, reward: 2.59526102949
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 2.595261029493114, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.60)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: right, reward: 0.801387434948
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.801387434947849, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent drove right instead of forward. (rewarded 0.80)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: left, reward: 2.11352517947
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 2.1135251794693426, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 2.11)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.97510836314
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.9751083631382753, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.98)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 0.270918908027
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.27091890802734486, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent drove right instead of forward. (rewarded 0.27)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 0.256390764451
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.2563907644514938, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent drove right instead of left. (rewarded 0.26)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: 0.50913434019
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.5091343401903584, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 0.51)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: left, reward: -40.4671661085
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -40.46716610845989, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.47)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: -0.512613070908
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'right'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': -0.512613070907755, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'right')
Agent drove right instead of left. (rewarded -0.51)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 2.33701428965
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 2.3370142896476773, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 2.34)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: -0.103643446229
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': -0.10364344622912391, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded -0.10)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: forward, reward: 0.944320561398
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 0.944320561398317, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded 0.94)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 0.816794532615
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.8167945326151826, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 0.82)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 15
\-------------------------
Environment.reset(): Trial set up with start = (7, 6), destination = (5, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 1.95482012932
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.954820129323445, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.95)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: 0.195496357469
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'left', None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 0.19549635746895566, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'left', None)
Agent drove left instead of right. (rewarded 0.20)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 1.14826318921
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.148263189209644, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.15)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: left, reward: -40.4760637536
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': -40.47606375355255, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.48)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 1.05280968695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.0528096869469767, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.05)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 1.64031729098
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.6403172909835142, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.64)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 1.07866659536
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.0786665953582308, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.08)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: -9.75965997617
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': -9.759659976166189, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -9.76)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.37132607775
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.37132607775184, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.37)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 1.07788886503
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.0778888650271325, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.08)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: 1.29078337908
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 1.2907833790780086, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 1.29)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.68479542459
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.6847954245909618, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.68)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.74430299118
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.744302991183811, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.74)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: 0.759984540286
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'right'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 0.7599845402858088, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'right')
Agent followed the waypoint forward. (rewarded 0.76)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: 2.3449209102
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 2.34492091019553, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.34)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: forward, reward: 0.799667939138
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 0.7996679391382824, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.80)
36% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 16
\-------------------------
Environment.reset(): Trial set up with start = (5, 7), destination = (8, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: right, reward: 1.31048304988
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.3104830498784927, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.31)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 0.997570764165
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'right'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.9975707641647471, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'right')
Agent followed the waypoint right. (rewarded 1.00)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 2.9104146682
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.9104146682030185, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.91)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 2.37727618188
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.3772761818835075, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.38)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.07224315809
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.072243158091496, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.07)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 2.15605750885
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.1560575088466156, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.16)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 1.89743087622
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.897430876224468, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent followed the waypoint forward. (rewarded 1.90)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 1.86183999303
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.861839993026521, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.86)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: 1.68741850859
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.6874185085856546, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 1.69)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: 1.15973844954
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.1597384495442138, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.16)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 17
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (5, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 1.09207272267
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.092072722671656, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.09)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 1.46956104195
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.469561041953938, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent followed the waypoint right. (rewarded 1.47)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 2.31987116396
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.319871163956265, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.32)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.07821439568
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.0782143956771364, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.08)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.75586935985
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.7558693598504034, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.76)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 0.0362852800365
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.0362852800365443, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.04)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: forward, reward: 1.70734894
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.707348939999015, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 1.71)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: left, reward: 0.962878303112
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 0.962878303112382, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent followed the waypoint left. (rewarded 0.96)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: left, reward: -9.84590117496
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -9.845901174962933, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.85)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 0.731121721334
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.7311217213338195, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.73)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 1.22927084947
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.229270849466785, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.23)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 1.66876276924
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.6687627692393119, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 1.67)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 0.237486502893
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 0.23748650289250328, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.24)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 1.39079891835
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.390798918351682, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 1.39)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 0.588355974926
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 0.5883559749262343, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent drove right instead of left. (rewarded 0.59)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 1.34729263383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.3472926338333877, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.35)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: -0.466790111279
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': -0.46679011127911796, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded -0.47)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: forward, reward: -0.240110496212
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -0.24011049621211844, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', None)
Agent drove forward instead of left. (rewarded -0.24)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: None, reward: -4.05596948766
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 2, 't': 18, 'action': None, 'reward': -4.055969487656171, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.06)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: forward, reward: 0.150760121727
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 0.1507601217265454, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 0.15)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 18
\-------------------------
Environment.reset(): Trial set up with start = (8, 7), destination = (4, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: -19.0400408536
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': -19.04004085361657, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.04)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: right, reward: 1.62413389481
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.6241338948066237, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.62)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 1.32886812321
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.3288681232078556, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove right instead of forward. (rewarded 1.33)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: forward, reward: 0.0178119323226
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 0.017811932322586488, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 0.02)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: left, reward: 2.7051233721
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 2.7051233720982766, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.71)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 1.14455630435
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.1445563043450988, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.14)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 1.02604468039
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.0260446803896233, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.03)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: -40.1938138406
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': -40.193813840642584, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.19)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 0.966545105394
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 0.9665451053935399, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.97)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 0.130148446458
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.13014844645772183, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent drove right instead of left. (rewarded 0.13)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: forward, reward: 0.821753364661
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 0.8217533646607763, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove forward instead of right. (rewarded 0.82)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: left, reward: -40.0684640255
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': -40.068464025503324, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.07)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 1.25625351648
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.2562535164796957, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.26)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 1.08636134076
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.0863613407643438, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.09)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 2.3618641878
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.3618641877979396, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.36)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 2.56725750332
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.567257503316051, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.57)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: right, reward: 1.31287007334
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 1.312870073343845, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.31)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: right, reward: 0.412242826844
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 0.4122428268443723, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.41)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 0.927359122501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 0.9273591225013866, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 0.93)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: right, reward: 0.870925421336
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 0.8709254213356941, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 0.87)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: left, reward: 0.462616108372
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 20, 'action': 'left', 'reward': 0.46261610837213896, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.46)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 0.838575376999
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 4, 't': 21, 'action': None, 'reward': 0.8385753769985211, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 0.84)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 1.6138443037
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 22, 'action': None, 'reward': 1.6138443036978396, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.61)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 0.52747674719
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 2, 't': 23, 'action': None, 'reward': 0.5274767471903201, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 0.53)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: 0.939192424947
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 1, 't': 24, 'action': 'right', 'reward': 0.9391924249465864, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.94)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 19
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (5, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.01031737957
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.0103173795745963, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.01)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 2.22416366869
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.2241636686924773, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.22)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 1.95418863747
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.9541886374663815, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 1.95)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.79170772069
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.791707720687229, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.79)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 1.77735089726
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.777350897261985, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.78)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 1.6217471381
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.621747138096065, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.62)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 2.27398418593
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.2739841859315764, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.27)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: forward, reward: 1.48215916521
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.4821591652135653, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.48)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 1.49920375115
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.499203751151175, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.50)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: forward, reward: -40.4514925471
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': -40.45149254709795, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.45)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.45734211955
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.4573421195501397, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 1.46)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: left, reward: -20.8761710174
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'right', None, 'forward'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': -20.87617101738716, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.88)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 0.669705963739
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.669705963739049, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 0.67)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: 2.08307970679
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 2.083079706791252, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.08)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 1.99330991228
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 1.9933099122829652, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.99)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 2.37688110556
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.376881105557172, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.38)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 0.942733900184
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 0.9427339001838542, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.94)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 1.17594807348
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 1.1759480734778278, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.18)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: 1.59683563756
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 1.5968356375591541, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.60)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: right, reward: 0.416357208932
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.41635720893243866, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove right instead of left. (rewarded 0.42)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 20
\-------------------------
Environment.reset(): Trial set up with start = (7, 6), destination = (4, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: forward, reward: -10.369983616
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -10.369983616011886, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -10.37)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 2.09836479501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.098364795006292, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.10)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 2.17609241067
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 2.176092410670308, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.18)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: forward, reward: 2.70316713354
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.70316713354171, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent followed the waypoint forward. (rewarded 2.70)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.40925632276
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'right'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.4092563227618924, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'right')
Agent properly idled at a red light. (rewarded 2.41)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: forward, reward: 2.83520997247
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.8352099724664193, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.84)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: right, reward: 2.55867358676
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.5586735867559796, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.56)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 1.02550230814
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.0255023081437686, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.03)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 2.54336050568
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.543360505682838, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.54)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 2.41396809353
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.4139680935288075, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.41)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 0.84897103772
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 0.8489710377203066, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.85)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: left, reward: 1.06024828024
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.0602482802411908, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent drove left instead of forward. (rewarded 1.06)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 2.59847998826
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'right'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 2.598479988260957, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'right')
Agent followed the waypoint right. (rewarded 2.60)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 1.49000939185
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', None), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.49000939185107, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 1.49)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: forward, reward: -10.3059313341
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': -10.305931334121135, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent attempted driving forward through a red light. (rewarded -10.31)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: forward, reward: -40.7761685472
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': -40.77616854716424, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.78)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 2.08558030194
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 9, 't': 16, 'action': None, 'reward': 2.085580301935658, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.09)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: 1.13724835267
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 1.1372483526651, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 1.14)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 0.933250236123
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'forward'), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 0.9332502361229584, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'forward')
Agent followed the waypoint right. (rewarded 0.93)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: 1.86276338797
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 1.8627633879678116, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.86)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: -0.378168041363
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 5, 't': 20, 'action': 'right', 'reward': -0.37816804136319315, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded -0.38)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: 1.73804232782
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 4, 't': 21, 'action': None, 'reward': 1.7380423278183998, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.74)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: 1.24838427386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 3, 't': 22, 'action': None, 'reward': 1.2483842738579154, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.25)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: 1.81163730941
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.8116373094141314, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.81)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: 0.497811054108
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.49781105410751736, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.50)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 1
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (5, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 2.71297810172
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.7129781017226713, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.71)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 2.22059581041
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.2205958104130588, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.22)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 2.29702268632
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 2.297022686319499, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.30)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 0.785382403972
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 0.7853824039724062, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent drove right instead of forward. (rewarded 0.79)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: left, reward: 2.03454042081
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 2.0345404208123052, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.03)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: left, reward: 1.76872091832
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.7687209183158352, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.77)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 2.14184446103
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.141844461027417, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.14)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 2.16289884031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.1628988403062985, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.16)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: left, reward: -10.6355562433
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': -10.635556243263597, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', None)
Agent attempted driving left through a red light. (rewarded -10.64)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: 1.5177559232
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.5177559231984203, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.52)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: right, reward: -0.127231654124
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': -0.12723165412353998, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded -0.13)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: left, reward: 0.249734937317
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 0.24973493731739815, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.25)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 0.859877001571
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 0.8598770015705255, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 0.86)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 1.85265030822
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'left'), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.85265030822219, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.85)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 0.732033651626
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 11, 't': 14, 'action': None, 'reward': 0.7320336516255368, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 0.73)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: left, reward: -40.4859882072
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 10, 't': 15, 'action': 'left', 'reward': -40.48598820723645, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.49)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: right, reward: 2.26936055864
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 2.269360558639143, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.27)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: right, reward: 0.315014077083
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 0.31501407708325546, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.32)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 0.862117520241
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 0.8621175202406199, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.86)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 0.790871175924
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 6, 't': 19, 'action': None, 'reward': 0.7908711759242939, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 0.79)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: right, reward: -20.9178941685
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 5, 't': 20, 'action': 'right', 'reward': -20.917894168524814, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.92)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: forward, reward: 0.408310831761
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 0.4083108317609069, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.41)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: right, reward: -0.72636234306
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': -0.7263623430597019, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded -0.73)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: left, reward: 1.58082946009
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 2, 't': 23, 'action': 'left', 'reward': 1.5808294600857662, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 1.58)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: -0.356520121948
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 24, 'action': 'right', 'reward': -0.3565201219482321, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded -0.36)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 2
\-------------------------
Environment.reset(): Trial set up with start = (7, 3), destination = (4, 7), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: forward, reward: 1.79705841687
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.7970584168691766, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.80)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 1.49109835455
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.4910983545486, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded 1.49)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: -4.61821155253
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'right'), 'deadline': 23, 't': 2, 'action': None, 'reward': -4.618211552531935, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.62)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: left, reward: 1.56469102942
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 1.5646910294233551, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 1.56)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: 1.44204433565
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 1.4420443356453274, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.44)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: left, reward: 1.33461725102
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.334617251019111, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 1.33)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 1.89161624134
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.8916162413446578, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.89)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 1.80927728595
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.809277285954558, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.81)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: right, reward: 1.95986242678
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.9598624267823466, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.96)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 2.70727797712
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 2.7072779771224207, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.71)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 0.936884538483
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 0.9368845384828872, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 0.94)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: forward, reward: 1.61339235602
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.6133923560248458, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent drove forward instead of right. (rewarded 1.61)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 1.29452996844
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.2945299684401574, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.29)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 1.73435672812
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.734356728118075, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.73)
44% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 3
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (1, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: 1.13080422521
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.1308042252056012, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 1.13)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 0.532163245639
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.5321632456389502, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', None)
Agent drove right instead of left. (rewarded 0.53)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: forward, reward: -10.5660858456
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': -10.566085845557284, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.57)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 1.32844414432
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.3284441443204784, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.33)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 1.74337520417
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.743375204171016, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 1.74)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: 1.85745262818
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.857452628180763, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.86)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: 1.81828271007
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.8182827100686194, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 1.82)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: 1.87355242493
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 1.8735524249268538, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent followed the waypoint left. (rewarded 1.87)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 1.83660013135
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.8366001313518026, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent drove right instead of left. (rewarded 1.84)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: left, reward: -39.7636683817
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'right', 'right'), 'deadline': 16, 't': 9, 'action': 'left', 'reward': -39.76366838167577, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.76)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 0.035233634329
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.03523363432895421, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent drove right instead of left. (rewarded 0.04)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: 1.78967537315
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.789675373149414, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove left instead of right. (rewarded 1.79)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: 0.88504048591
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', None), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 0.8850404859104095, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', None)
Agent followed the waypoint forward. (rewarded 0.89)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.27419595259
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 2.2741959525864566, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.27)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 0.830302605989
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 0.8303026059894429, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.83)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 2.21762826191
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 2.2176282619095335, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.22)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 0.628163765668
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 0.6281637656678444, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 0.63)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: -0.355642039006
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': -0.35564203900618385, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded -0.36)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: forward, reward: 1.43902511496
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': 1.439025114956936, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.44)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 1.35892493681
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'forward'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 1.3589249368128113, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'forward')
Agent drove right instead of left. (rewarded 1.36)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: left, reward: -0.532880976162
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 5, 't': 20, 'action': 'left', 'reward': -0.5328809761619813, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove left instead of right. (rewarded -0.53)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: 2.14081993404
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 2.1408199340440053, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.14)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 0.484370477323
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 3, 't': 22, 'action': None, 'reward': 0.48437047732302285, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 0.48)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.73314840416
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.7331484041559804, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.73)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 0.320126146245
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 1, 't': 24, 'action': 'right', 'reward': 0.32012614624485347, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent drove right instead of forward. (rewarded 0.32)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 4
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (1, 2), deadline = 30
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: left, reward: 2.81725578384
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 2.8172557838381485, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.82)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: None, reward: 1.89530776198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.895307761978669, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.90)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: forward, reward: -40.3485350437
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': -40.34853504367269, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.35)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: left, reward: -10.7953855998
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 27, 't': 3, 'action': 'left', 'reward': -10.79538559980644, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent attempted driving left through a red light. (rewarded -10.80)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: None, reward: 2.43459923616
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.434599236157384, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.43)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: None, reward: 1.85407056674
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.8540705667392263, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.85)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: forward, reward: 0.526091369908
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 0.5260913699081556, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.53)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: -20.8329200221
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 23, 't': 7, 'action': 'right', 'reward': -20.832920022076134, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.83)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 1.90286325363
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 1.9028632536284618, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.90)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: left, reward: -39.7239011976
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': -39.723901197610736, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.72)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.8656564113
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.8656564112971505, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.87)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: right, reward: 1.69778759928
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 1.6977875992847422, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.70)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 2.66122350157
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 2.6612235015680223, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.66)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 0.848705241406
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 0.8487052414056349, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 0.85)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 0.846117035336
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 0.8461170353356391, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.85)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: 1.49570936809
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.495709368085667, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.50)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.11854759955
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 14, 't': 16, 'action': None, 'reward': 2.118547599548932, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.12)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 1.00868172805
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'forward'), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 1.0086817280519011, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'forward')
Agent drove right instead of left. (rewarded 1.01)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: forward, reward: 1.50860724447
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 12, 't': 18, 'action': 'forward', 'reward': 1.5086072444708174, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove forward instead of right. (rewarded 1.51)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 2.47652452136
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 2.476524521359705, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.48)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 0.846405485068
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 0.8464054850684364, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.85)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 2.21176023314
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 9, 't': 21, 'action': None, 'reward': 2.2117602331350934, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.21)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: -20.5623473592
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', 'left'), 'deadline': 8, 't': 22, 'action': 'right', 'reward': -20.56234735915224, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.56)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 1.81753321121
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 7, 't': 23, 'action': None, 'reward': 1.817533211209175, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.82)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: -19.3959141322
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 6, 't': 24, 'action': 'right', 'reward': -19.395914132224082, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.40)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: forward, reward: -40.4854169837
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', 'left'), 'deadline': 5, 't': 25, 'action': 'forward', 'reward': -40.48541698366536, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.49)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: forward, reward: 0.741949909512
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 26, 'action': 'forward', 'reward': 0.7419499095120841, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.74)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: -0.241519449337
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 3, 't': 27, 'action': 'right', 'reward': -0.24151944933698744, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove right instead of left. (rewarded -0.24)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.42099352003
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 2, 't': 28, 'action': None, 'reward': 1.4209935200288817, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.42)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.4126907038
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 1, 't': 29, 'action': None, 'reward': 1.4126907037968621, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.41)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 5
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (4, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: -5.6778297728
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'right', 'right'), 'deadline': 20, 't': 0, 'action': None, 'reward': -5.677829772804305, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.68)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: right, reward: 2.46383199658
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.4638319965784423, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.46)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 1.58767294314
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.5876729431373833, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.59)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: right, reward: 0.205336231592
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 0.20533623159171854, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.21)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: left, reward: 1.35987685851
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.3598768585110292, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 1.36)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: left, reward: 1.62682560775
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.6268256077464172, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.63)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 1.21879356425
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.2187935642474306, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.22)
65% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 6
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (8, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: right, reward: 2.44832231259
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.4483223125916695, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.45)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 0.30903174922
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.30903174922041743, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove right instead of forward. (rewarded 0.31)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 2.55403571793
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.5540357179274844, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.55)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: forward, reward: 0.460229489164
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 0.4602294891635833, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.46)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 0.966645841463
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 0.9666458414629722, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 0.97)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 0.775038489885
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.7750384898854634, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.78)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: -40.6806519406
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -40.68065194064458, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.68)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 2.33145264864
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.3314526486377725, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.33)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 2.02308407634
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.0230840763401376, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.02)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: forward, reward: -40.5506360637
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': -40.55063606369617, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.55)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 1.0203017269
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.0203017269017207, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 1.02)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 1.04058155583
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.0405815558258595, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.04)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 0.619502697102
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.6195026971020487, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.62)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: left, reward: 0.938043396972
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.9380433969722454, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove left instead of right. (rewarded 0.94)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 0.106145043655
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 0.1061450436549577, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove right instead of left. (rewarded 0.11)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 0.0403312089693
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 0.040331208969327736, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.04)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 0.994254222806
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.9942542228055748, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.99)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 0.408334106485
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.4083341064845014, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 0.41)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 1.22149864455
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'right'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 1.221498644554707, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'right')
Agent drove right instead of left. (rewarded 1.22)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: 1.22516638173
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 1.2251663817332117, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.23)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 7
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (2, 7), deadline = 30
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 1.02494563749
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.024945637485326, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.02)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 2.4711448051
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.4711448051046734, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.47)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: right, reward: 0.636614222387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 0.6366142223874679, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent drove right instead of left. (rewarded 0.64)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: right, reward: 1.83688084351
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 1.8368808435071007, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.84)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 0.75117296167
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 0.7511729616703112, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent drove right instead of forward. (rewarded 0.75)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: 0.244539918897
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 0.24453991889729088, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.24)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: left, reward: 1.77597678718
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 1.7759767871787635, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.78)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 1.23116311226
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.2311631122626254, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 1.23)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: right, reward: 0.556407561429
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 0.5564075614286327, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.56)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: -5.78440164174
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 21, 't': 9, 'action': None, 'reward': -5.784401641741491, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.78)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: left, reward: 1.85793326478
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 1.8579332647796218, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.86)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 2.37351505424
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.373515054237988, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.37)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 2.66236097433
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', None), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.662360974333156, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 2.66)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 1.86622980761
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 13, 'action': None, 'reward': 1.866229807613109, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.87)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 1.64754713842
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.6475471384173264, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.65)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 2.39938469701
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': None, 'reward': 2.3993846970128265, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.40)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: forward, reward: 2.18170125676
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': 2.1817012567580294, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent followed the waypoint forward. (rewarded 2.18)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 0.406445106041
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 13, 't': 17, 'action': None, 'reward': 0.4064451060410881, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.41)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: -0.129170465609
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 12, 't': 18, 'action': 'right', 'reward': -0.1291704656088153, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove right instead of forward. (rewarded -0.13)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 2.45330709814
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 11, 't': 19, 'action': None, 'reward': 2.4533070981361615, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.45)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.42529452669
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 10, 't': 20, 'action': None, 'reward': 1.4252945266885988, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.43)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: 1.53849044013
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 21, 'action': 'left', 'reward': 1.5384904401302368, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.54)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 0.584965921257
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 8, 't': 22, 'action': None, 'reward': 0.5849659212565339, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 0.58)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 0.0552700051163
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 0.05527000511634339, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent drove right instead of forward. (rewarded 0.06)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 0.560013241588
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 0.560013241587602, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.56)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 0.949428051413
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 0.9494280514129549, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.95)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 1.56607862209
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 4, 't': 26, 'action': 'right', 'reward': 1.5660786220890843, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.57)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: -0.559053424902
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'right'), 'deadline': 3, 't': 27, 'action': 'right', 'reward': -0.5590534249021224, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'right')
Agent drove right instead of forward. (rewarded -0.56)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: left, reward: 1.06256098757
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 28, 'action': 'left', 'reward': 1.0625609875741093, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.06)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 0.304230019607
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 1, 't': 29, 'action': 'right', 'reward': 0.3042300196073211, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 0.30)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 8
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (1, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 1.40378358904
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.4037835890446362, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.40)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: -10.8862445063
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'forward'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -10.886244506345534, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -10.89)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: right, reward: 1.92379832653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.9237983265260157, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.92)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 0.324582991331
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 0.3245829913310273, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove right instead of left. (rewarded 0.32)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 2.67662485687
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.676624856874076, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.68)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: -20.94956982
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': -20.949569819972265, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.95)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: forward, reward: -10.01082133
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -10.010821329961823, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.01)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: left, reward: 0.792776223034
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 0.7927762230340119, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove left instead of right. (rewarded 0.79)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 1.6734834216
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.6734834215966308, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.67)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: right, reward: 0.709935239476
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.7099352394760743, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.71)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 2.54800523655
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.548005236546299, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.55)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 0.850036424082
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 0.850036424081585, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.85)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 0.909730828296
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.9097308282964429, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.91)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: left, reward: 2.4718299244
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 2.4718299243985733, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', 'left')
Agent followed the waypoint left. (rewarded 2.47)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: left, reward: 0.0308603120982
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 0.030860312098155762, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.03)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: right, reward: 1.40883827971
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.408838279713164, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.41)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 2.22479185304
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 4, 't': 16, 'action': None, 'reward': 2.2247918530385986, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.22)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 1.33366687916
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.3336668791583475, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.33)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 1.19988222547
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.1998822254700083, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.20)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 0.654139669978
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.6541396699784443, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.65)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 9
\-------------------------
Environment.reset(): Trial set up with start = (8, 7), destination = (2, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: 2.20625910005
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.2062591000473244, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'forward')
Agent followed the waypoint forward. (rewarded 2.21)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 1.32838068932
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.3283806893163281, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded 1.33)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 2.07172225218
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.071722252182297, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.07)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 0.993828065745
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 0.9938280657449612, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.99)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 0.997694362282
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.9976943622823055, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 1.00)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 1.65924997087
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.6592499708693023, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.66)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 1.13703848223
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.1370384822266721, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.14)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 1.0524080486
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.052408048603179, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.05)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 1.89579227564
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.895792275644966, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.90)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: left, reward: 1.15372082412
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.1537208241224575, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 1.15)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: left, reward: 0.71475081495
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 0.7147508149498735, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent drove left instead of right. (rewarded 0.71)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.23716942679
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.2371694267866942, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.24)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 0.977832015937
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.9778320159374325, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.98)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 2.54325543549
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 2.5432554354910852, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.54)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: -10.0815706523
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': -10.081570652304224, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.08)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 1.75940628073
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.7594062807332735, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.76)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: -0.0822450061912
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 4, 't': 16, 'action': 'left', 'reward': -0.08224500619118635, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent drove left instead of forward. (rewarded -0.08)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: -40.2127513493
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -40.212751349289725, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.21)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 2.15165433222
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 2.1516543322204784, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.15)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 0.874630878213
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.8746308782132642, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, 'right', None)
Agent drove right instead of left. (rewarded 0.87)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 10
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (4, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 0.975299216982
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.9752992169818886, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent drove right instead of forward. (rewarded 0.98)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 2.53785232411
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.5378523241088864, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.54)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: forward, reward: -39.4063523617
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': -39.40635236170391, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.41)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.03397242134
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.0339724213397123, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.03)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 2.3166834136
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.3166834136019077, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.32)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: 0.449084384157
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.4490843841568579, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'left')
Agent drove right instead of left. (rewarded 0.45)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: left, reward: 2.45685219932
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.4568521993222188, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.46)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: forward, reward: -40.8264968222
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -40.82649682224445, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.83)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 1.58986331211
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.5898633121128642, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.59)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 2.74329458059
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.743294580589077, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.74)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 1.59605517297
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.5960551729741188, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.60)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 1.80047048304
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.8004704830437244, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.80)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: -0.320635606258
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': -0.3206356062582745, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded -0.32)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: left, reward: 0.843598662833
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.8435986628325025, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.84)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: -39.1308359641
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'right'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': -39.13083596406945, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'right')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.13)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: left, reward: -40.0640405829
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -40.064040582926836, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.06)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 0.525191149158
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 0.5251911491581207, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.53)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 2.18379560433
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 2.1837956043280977, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.18)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.83177838555
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.831778385545473, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 1.83)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.13974736484
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.1397473648440788, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.14)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
Simulation ended. . .
<matplotlib.figure.Figure at 0xdc449e8>
In [23]:
# self.epsilon = (self.a)**self.t
# Load the 'sim_improved-learning' file from the improved Q-Learning simulation
vs.plot_trials('sim_improved-learning.csv')
In [24]:
%run smartcab\agent.py
#可以观察到可视化程序的各部分的功能
/-------------------------
| Training trial 1
\-------------------------
Environment.reset(): Trial set up with start = (3, 2), destination = (8, 3), deadline = 20
Simulating trial. . .
epsilon = 1.0000; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: forward, reward: 0.880473857701
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 0.880473857701084, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.88)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: 1.49331264439
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.493312644388895, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 1.49)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.00419912302
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.0041991230212999, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.00)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: -9.84635706069
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -9.846357060686115, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.85)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: -10.8863742096
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -10.886374209569931, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -10.89)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.59223995465
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.592239954649814, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.59)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 2.48533044263
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 2.4853304426267346, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.49)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: left, reward: -40.0079708956
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'right'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', 'right'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': -40.00797089556902, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.01)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 2.02854866065
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'right'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.0285486606474947, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 2.03)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: left, reward: 1.07989954414
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.0798995441389352, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.08)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: -5.12839773683
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': -5.128397736831089, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.13)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 0.832366552692
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'forward'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.8323665526921253, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'forward')
Agent followed the waypoint right. (rewarded 0.83)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.97301987622
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.9730198762157922, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.97)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 0.216914527118
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.21691452711835646, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.22)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.29512625969
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.2951262596855697, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 2.30)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: left, reward: -9.35105958713
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -9.351059587129361, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.35)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: left, reward: -9.73568890377
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'left', 'reward': -9.73568890377447, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.74)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: -0.525236496496
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': None, 'reward': -0.525236496496014, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.53)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: -0.42830779326
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': -0.428307793259596, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded -0.43)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 0.680764678299
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.6807646782992395, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.68)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 2
\-------------------------
Environment.reset(): Trial set up with start = (4, 3), destination = (1, 6), deadline = 30
Simulating trial. . .
epsilon = 0.2500; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: right, reward: 2.20216397394
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 2.20216397393799, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.20)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: 1.24836074716
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 1.2483607471622395, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent followed the waypoint forward. (rewarded 1.25)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: -5.35406041935
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 28, 't': 2, 'action': None, 'reward': -5.354060419351994, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.35)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 0.152741695979
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 0.15274169597935072, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent drove right instead of forward. (rewarded 0.15)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: -5.87729695923
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 26, 't': 4, 'action': None, 'reward': -5.877296959227808, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.88)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: left, reward: 2.18570838287
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 2.1857083828697736, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 2.19)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: 0.580294810718
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 0.5802948107179814, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove forward instead of right. (rewarded 0.58)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: left, reward: 1.22333203429
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 1.223332034291575, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 1.22)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: forward, reward: -10.526202019
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': -10.526202019030013, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.53)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 2.86061913579
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 21, 't': 9, 'action': None, 'reward': 2.8606191357928594, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.86)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 2.24597115859
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.245971158585294, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.25)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: forward, reward: 0.850741695686
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 0.8507416956857011, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.85)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: left, reward: 2.79667571961
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 18, 't': 12, 'action': 'left', 'reward': 2.796675719613787, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.80)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: left, reward: -0.0868173069463
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': -0.08681730694629053, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded -0.09)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: 2.30939032623
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': 2.3093903262290976, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.31)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: -9.67345776889
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': -9.673457768888653, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.67)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: -9.04962218562
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 14, 't': 16, 'action': 'left', 'reward': -9.049622185615393, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent attempted driving left through a red light. (rewarded -9.05)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 0.660790727953
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 0.6607907279530123, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.66)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: -19.0737906269
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 12, 't': 18, 'action': 'left', 'reward': -19.07379062692848, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.07)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 0.950671206477
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 0.9506712064771498, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove right instead of left. (rewarded 0.95)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 2.50374007764
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 10, 't': 20, 'action': None, 'reward': 2.503740077642851, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.50)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: left, reward: 0.944493668532
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'left', 'reward': 0.9444936685323387, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.94)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 1.47342013546
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 1.4734201354552405, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.47)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 0.646643491205
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 0.6466434912045163, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.65)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 2.21688063006
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 6, 't': 24, 'action': 'forward', 'reward': 2.2168806300617474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.22)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 0.79351961086
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 0.7935196108604965, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 0.79)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: -0.257672492887
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 4, 't': 26, 'action': 'right', 'reward': -0.2576724928868148, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded -0.26)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: -0.656028375144
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'right'), 'deadline': 3, 't': 27, 'action': 'right', 'reward': -0.6560283751437344, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'right')
Agent drove right instead of left. (rewarded -0.66)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: forward, reward: 0.11881922258
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 2, 't': 28, 'action': 'forward', 'reward': 0.11881922257991928, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.12)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: left, reward: -10.013193459
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 1, 't': 29, 'action': 'left', 'reward': -10.013193458965944, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.01)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 3
\-------------------------
Environment.reset(): Trial set up with start = (7, 6), destination = (2, 5), deadline = 20
Simulating trial. . .
epsilon = 0.1111; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: -20.3884526056
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': -20.388452605586085, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.39)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 1.90902723891
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.909027238909687, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.91)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 1.50037915273
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.5003791527252976, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.50)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 1.92119813738
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.9211981373822284, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.92)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 2.5032222084
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.5032222083975135, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.50)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 2.90948174399
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 2.9094817439883767, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.91)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 1.427436288
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.4274362880026699, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.43)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 1.44532724642
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.4453272464161282, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent drove forward instead of left. (rewarded 1.45)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: -4.40648382741
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'right', 'right', None), 'deadline': 12, 't': 8, 'action': None, 'reward': -4.406483827405129, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.41)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: 1.38579488214
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.3857948821384958, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.39)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: -10.8117611993
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': -10.811761199333464, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.81)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 1.0819658049
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.08196580490381, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.08)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 2.15411763098
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.1541176309792713, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.15)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: 0.607474353788
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.607474353788002, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 0.61)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 1.47535998672
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.4753599867182723, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.48)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: 0.980916754156
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 0.9809167541560526, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded 0.98)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 0.581684426777
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'left'), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.5816844267773702, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 0.58)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 2.31867578397
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'left'), 'deadline': 3, 't': 17, 'action': None, 'reward': 2.3186757839656194, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.32)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: -9.36812583638
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -9.368125836381006, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent attempted driving left through a red light. (rewarded -9.37)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 0.96572713537
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.9657271353697694, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 0.97)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 4
\-------------------------
Environment.reset(): Trial set up with start = (5, 7), destination = (1, 5), deadline = 30
Simulating trial. . .
epsilon = 0.0625; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0625; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0625; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: forward, reward: -10.9839389374
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', 'forward'), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': -10.983938937388253, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -10.98)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: forward, reward: -9.63869388031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': -9.638693880306162, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -9.64)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: right, reward: 0.614119414923
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 0.6141194149231247, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 0.61)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: right, reward: 1.7497857738
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 1.749785773800248, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 1.75)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: -5.30521733467
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 26, 't': 4, 'action': None, 'reward': -5.305217334666161, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.31)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 1.09090598265
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.0909059826548355, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.09)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: left, reward: 1.9901318542
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 1.990131854198459, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent followed the waypoint left. (rewarded 1.99)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 0.442183272915
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 0.4421832729150824, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.44)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.91802558276
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.9180255827649908, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.92)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 0.510180757296
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 0.5101807572957606, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.51)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: forward, reward: 1.86668655851
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 1.8666865585146053, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.87)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: left, reward: 0.462387422007
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 0.4623874220070162, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove left instead of forward. (rewarded 0.46)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: None, reward: 2.56208173617
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.5620817361733565, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.56)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: left, reward: 1.61085095076
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 1.6108509507604214, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 1.61)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: left, reward: -9.74971173783
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', 'forward'), 'deadline': 16, 't': 14, 'action': 'left', 'reward': -9.749711737833794, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -9.75)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 0.704388350819
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 0.7043883508191849, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.70)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 1.74317539941
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 1.743175399411899, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.74)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: right, reward: 0.861060403543
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 0.8610604035434866, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent drove right instead of forward. (rewarded 0.86)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: 2.57652596209
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 12, 't': 18, 'action': 'left', 'reward': 2.5765259620893817, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.58)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: -40.3600652367
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 11, 't': 19, 'action': 'left', 'reward': -40.36006523666612, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.36)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 0.725843057145
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 0.7258430571454442, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.73)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: 0.102405272537
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 0.10240527253703713, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.10)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 0.555863508424
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 0.5558635084238113, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 0.56)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 0.485064426692
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 23, 'action': None, 'reward': 0.48506442669167926, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.49)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 2.07051164155
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 6, 't': 24, 'action': None, 'reward': 2.07051164154685, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.07)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: left, reward: 0.797995961727
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 5, 't': 25, 'action': 'left', 'reward': 0.7979959617267043, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.80)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: left, reward: 0.633327108004
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 4, 't': 26, 'action': 'left', 'reward': 0.6333271080043636, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent drove left instead of right. (rewarded 0.63)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 0.430304837315
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 3, 't': 27, 'action': None, 'reward': 0.4303048373149494, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.43)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 1.73451975998
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 2, 't': 28, 'action': None, 'reward': 1.7345197599811593, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.73)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: forward, reward: -0.569095890382
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 1, 't': 29, 'action': 'forward', 'reward': -0.569095890382429, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded -0.57)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 5
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (7, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0400; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0400; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0400; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.60652051399
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.6065205139949386, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.61)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.78907793338
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.7890779333830773, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 1.79)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.09914506329
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.0991450632938737, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.10)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.82891565965
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.828915659653607, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.83)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 1.59953684252
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.599536842523475, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 1.60)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: left, reward: 0.806202436409
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 0.8062024364093212, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.81)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 1.86751665536
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.8675166553571398, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 1.87)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 1.34204844992
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.3420484499214758, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove right instead of left. (rewarded 1.34)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 2.21251303972
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.2125130397169293, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.21)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 1.56178528841
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.5617852884089674, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.56)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: left, reward: 0.795730736712
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 0.7957307367120902, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.80)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: -0.103368512027
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': -0.10336851202680508, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded -0.10)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: left, reward: 1.48473230849
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 1.484732308486306, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent followed the waypoint left. (rewarded 1.48)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: 1.65692660498
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.656926604981485, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent followed the waypoint forward. (rewarded 1.66)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: -9.11174779086
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': -9.111747790856331, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.11)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: left, reward: -9.64208696402
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -9.642086964021415, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.64)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 0.865169602423
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.8651696024225539, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.87)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 0.122178520297
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.12217852029668774, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.12)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.67895367002
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.678953670019627, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.68)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 0.179966748204
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.1799667482036782, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 0.18)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 6
\-------------------------
Environment.reset(): Trial set up with start = (3, 4), destination = (2, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0278; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: 2.82536848874
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.8253684887406925, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.83)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: -20.9037324126
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': -20.903732412556383, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.90)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: 2.75950010019
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.759500100193586, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.76)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: left, reward: 0.768571806507
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 0.7685718065070825, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.77)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 2.90495368029
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.9049536802937155, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.90)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: forward, reward: 0.44848768349
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 0.44848768349002754, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.45)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: forward, reward: 0.835541974631
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 0.8355419746306292, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.84)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: forward, reward: -40.6939175753
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -40.6939175753153, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.69)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: right, reward: -0.0411137746463
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': -0.04111377464629029, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded -0.04)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 1.45120399454
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.4512039945354864, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 1.45)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 0.108835581635
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.10883558163483309, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.11)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: 0.683124290838
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.6831242908375821, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.68)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: right, reward: 1.08138929864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.0813892986421267, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 1.08)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: 1.73298469318
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.732984693175508, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.73)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 1.08919935496
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.0891993549582537, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 1.09)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: left, reward: -20.6254368611
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'forward'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -20.62543686112639, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.63)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: left, reward: 0.812965757852
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 0.8129657578519704, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove left instead of forward. (rewarded 0.81)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: forward, reward: -0.566361242726
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -0.5663612427258701, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove forward instead of right. (rewarded -0.57)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: right, reward: 0.29450932975
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 0.2945093297500456, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 0.29)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 0.790680207545
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.790680207544573, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.79)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 7
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (5, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0204; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0204; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: -19.2210673385
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': -19.22106733850251, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.22)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: left, reward: -9.34601176313
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -9.346011763129276, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.35)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.70731167091
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.7073116709138119, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.71)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 2.47852179267
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.4785217926692362, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.48)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 0.94468109952
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 0.944681099520374, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.94)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 2.36518561459
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.3651856145863253, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.37)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: left, reward: 1.3760525832
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.3760525831973809, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent drove left instead of right. (rewarded 1.38)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: -9.32033810525
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -9.320338105247778, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.32)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 0.178161486465
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.17816148646472174, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.18)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 0.42439526636
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.42439526635994396, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 0.42)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: left, reward: -40.8671489689
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'forward'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -40.86714896891975, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.87)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.35089883788
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.3508988378815534, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 2.35)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 1.46066977796
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.4606697779569766, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 1.46)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 0.735730756351
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.7357307563514172, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 0.74)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.28325777984
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.28325777984019, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.28)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.15734604137
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.1573460413729617, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.16)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 0.929113727186
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.9291137271863639, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.93)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: -4.80760383542
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'forward'), 'deadline': 3, 't': 17, 'action': None, 'reward': -4.807603835423197, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.81)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 0.821787869831
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 0.8217878698314933, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.82)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 1.11278782567
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 1.1127878256679191, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.11)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 8
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (4, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0156; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0156; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0156; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: 2.03931466607
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.039314666071871, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.04)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: 1.4444021215
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.4444021215032774, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', None)
Agent drove forward instead of left. (rewarded 1.44)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: 0.672425082611
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.672425082611484, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.67)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 1.46827653095
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.4682765309502457, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.47)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: 1.67456010757
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.6745601075669936, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.67)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 1.80448284851
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.8044828485142173, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.80)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: left, reward: -0.0529235628918
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': -0.05292356289177658, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded -0.05)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 2.27106533904
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 2.271065339038488, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.27)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 1.57226775021
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.5722677502098465, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.57)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 1.44690606626
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.446906066255993, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.45)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 2.23525965797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.235259657965467, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.24)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: forward, reward: 0.775464528159
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.7754645281585433, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove forward instead of right. (rewarded 0.78)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: forward, reward: -0.120422693344
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': -0.12042269334350642, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded -0.12)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: -20.8745889143
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': -20.87458891434821, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.87)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 1.04445073046
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.0444507304556854, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.04)
25% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 9
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (5, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0123; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 0.668956470754
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.66895647075419, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.67)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: -39.1662058017
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'right'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -39.16620580167021, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'right')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.17)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.12762144534
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.1276214453421785, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.13)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: left, reward: 0.144937536343
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 0.1449375363428519, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.14)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: left, reward: 0.521060834014
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 0.5210608340142054, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.52)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 1.81456321145
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.8145632114474803, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.81)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 1.36024641134
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.3602464113380686, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.36)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: 1.53165179163
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.5316517916251775, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.53)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: -40.7769892479
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -40.77698924789011, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.78)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: -40.1855222602
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': -40.18552226016359, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.19)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: forward, reward: 1.12844431781
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.1284443178128485, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.13)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 0.550001593362
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.5500015933623514, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent drove right instead of forward. (rewarded 0.55)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 0.477471341423
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.4774713414227596, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.48)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: -5.21219542178
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'forward'), 'deadline': 7, 't': 13, 'action': None, 'reward': -5.212195421783809, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.21)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: -0.065111542432
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': -0.06511154243202466, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent drove forward instead of right. (rewarded -0.07)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: -4.25409279764
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 5, 't': 15, 'action': None, 'reward': -4.254092797638359, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.25)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: -19.8402538987
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'right'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': -19.840253898665274, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'right')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.84)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.38871489619
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.388714896187057, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.39)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.10027430214
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.1002743021355734, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.10)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.77476070738
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.7747607073783873, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.77)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 10
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (7, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0100; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0100; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0100; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0100; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: left, reward: 0.181190625081
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 0.18119062508076889, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.18)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: right, reward: 2.87410401079
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.8741040107945586, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.87)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: left, reward: 1.08365939372
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.0836593937168995, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove left instead of forward. (rewarded 1.08)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: right, reward: 2.61089961317
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 2.610899613171574, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.61)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: -4.55931876117
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': -4.55931876117268, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.56)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 0.290484283713
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.2904842837129762, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 0.29)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: forward, reward: -10.1324183949
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -10.132418394937757, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -10.13)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.09373345471
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.0937334547144333, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.09)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.66342112286
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.6634211228562095, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 1.66)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.259706406
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.2597064059995942, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.26)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.91457727638
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.914577276379738, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.91)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: forward, reward: 0.175208463947
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.1752084639469409, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent drove forward instead of left. (rewarded 0.18)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: left, reward: 1.25728315571
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 1.2572831557102784, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 1.26)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 0.836547731796
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.8365477317964786, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 0.84)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: 1.4564550791
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 1.456455079100515, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 1.46)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 0.136743310997
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 0.13674331099715098, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.14)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 0.516259248249
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.516259248249191, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.52)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 2.11247664945
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 2.1124766494523124, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.11)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 0.300835110156
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 0.30083511015609354, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.30)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 0.467343135934
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.4673431359341925, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 0.47)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 11
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (6, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0083; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 2.49521482061
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.495214820608131, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.50)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: left, reward: -19.8885351446
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -19.888535144598567, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.89)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: left, reward: 1.56700376596
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.5670037659609362, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.57)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 1.24322220682
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.2432222068238987, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.24)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 2.46764191993
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.467641919925126, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.47)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: 1.36073557929
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.3607355792877192, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.36)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: right, reward: 0.680305696417
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.6803056964173553, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.68)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: forward, reward: 1.45726492696
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.4572649269560203, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.46)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: 0.749800617481
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.7498006174810968, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 0.75)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: 0.768393349683
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 0.7683933496827777, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove left instead of right. (rewarded 0.77)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 0.0515986883103
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.05159868831033754, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove right instead of left. (rewarded 0.05)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: left, reward: 0.823216139996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 0.8232161399960012, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.82)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: left, reward: -40.4414435384
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'right'), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -40.441443538380554, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.44)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: 0.273647034607
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.27364703460669826, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 0.27)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 0.871709949031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.871709949030911, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 0.87)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 0.837410423563
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.8374104235633293, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.84)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 0.865690627256
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.8656906272555702, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent drove right instead of forward. (rewarded 0.87)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 1.00329964745
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 1.0032996474534612, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.00)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: -0.040437472327
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': -0.04043747232697603, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded -0.04)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 0.408729607239
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.4087296072389426, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 0.41)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 12
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (2, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0069; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0069; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: -4.40305429269
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': -4.403054292689891, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.40)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 1.09658959615
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.0965895961486356, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.10)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: -5.31972792515
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 18, 't': 2, 'action': None, 'reward': -5.319727925145343, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.32)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: forward, reward: -39.2198038091
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': -39.21980380909683, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.22)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 0.767388296198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.767388296197765, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.77)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 1.10832075242
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.1083207524249314, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.11)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: left, reward: 0.286054148957
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 0.2860541489573174, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove left instead of right. (rewarded 0.29)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 0.901738628474
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 0.901738628474414, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.90)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 0.630542978176
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.6305429781757083, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.63)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: 0.993176259543
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.9931762595426967, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 0.99)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 2.50504725118
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.505047251177894, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.51)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 2.23076900558
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.2307690055821183, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.23)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.9164689146
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.9164689146035592, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.92)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: -20.3707129664
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'forward'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': -20.370712966433658, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.37)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 1.68428227872
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 1.6842822787226361, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.68)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: forward, reward: 1.74465225628
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 1.7446522562847662, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.74)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.07293774875
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.0729377487545864, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.07)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 0.670200824751
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.6702008247505021, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 0.67)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: -0.654606673416
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': -0.6546066734161609, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded -0.65)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: left, reward: -39.19925559
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': -39.19925559001815, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.20)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 13
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (4, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0059; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 2.30581305688
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.3058130568778683, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.31)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: 1.34152508486
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.3415250848635927, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.34)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 1.50872009269
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.5087200926949342, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.51)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: None, reward: 1.59389088757
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.5938908875741922, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.59)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: 1.36608199734
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.3660819973386769, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.37)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 0.605792652027
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.6057926520267105, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.61)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.12225711219
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.1222571121932534, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.12)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 1.20643959115
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.2064395911486414, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 1.21)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 2.22873848562
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.2287384856214367, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.23)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 0.825001162448
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.8250011624482632, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.83)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: forward, reward: 0.993168435445
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 0.9931684354447001, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 0.99)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 14
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (3, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0051; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0051; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0051; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0051; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 0.847304406541
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.8473044065407473, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent drove right instead of forward. (rewarded 0.85)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: right, reward: 1.12785349079
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.1278534907867406, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 1.13)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.50537804842
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.5053780484182782, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.51)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.15876998816
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.1587699881597593, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.16)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.70485540147
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.7048554014724364, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.70)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: -39.2386360958
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': -39.23863609580054, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.24)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 0.941365222867
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.9413652228670704, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.94)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: forward, reward: 0.285217653284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 0.28521765328390325, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.29)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: None, reward: 1.61756426196
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.6175642619588733, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.62)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 1.49709860058
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.4970986005782283, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.50)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: 1.07849949817
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.0784994981713392, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 1.08)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: left, reward: 0.29252323262
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 0.2925232326201421, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.29)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 0.810728810059
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.8107288100594812, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 0.81)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: left, reward: 0.0898621778693
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.08986217786928508, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.09)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 0.611007148299
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 0.6110071482986136, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.61)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 0.51889885317
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.5188988531697325, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 0.52)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 1.30176071801
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.3017607180097208, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.30)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: left, reward: -0.67067857481
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': -0.6706785748098444, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent drove left instead of right. (rewarded -0.67)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: -0.464327523565
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': -0.46432752356525464, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded -0.46)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: right, reward: -0.331230888136
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': -0.33123088813570656, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded -0.33)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 15
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (8, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0044; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0044; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0044; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 2.97780133713
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.977801337126066, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.98)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.87087813429
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.8708781342881133, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.87)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.24558568505
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.2455856850532805, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.25)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.5035655322
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.5035655321974217, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.50)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 0.294563144227
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 0.2945631442268347, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 0.29)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: forward, reward: 1.03353993492
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.0335399349222132, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.03)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 0.85229945527
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.8522994552695288, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.85)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: 1.58889399601
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.5888939960140782, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.59)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: left, reward: 1.93614554676
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 1.9361455467607696, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.94)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 1.60524690512
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.6052469051245726, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.61)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: left, reward: 1.26778890616
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 1.267788906159331, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent drove left instead of right. (rewarded 1.27)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: right, reward: 1.24291004684
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.242910046842428, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.24)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: -5.00129327346
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'right'), 'deadline': 13, 't': 12, 'action': None, 'reward': -5.001293273456507, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.00)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: left, reward: -10.5325168915
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': -10.53251689145766, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.53)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: 1.75607646657
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.756076466572188, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.76)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: 2.09632982931
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.0963298293112373, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.10)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: right, reward: 0.532761256911
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 0.5327612569112331, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 0.53)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: -0.00615477022139
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': -0.0061547702213948785, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded -0.01)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: 0.751984863513
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 7, 't': 18, 'action': 'left', 'reward': 0.7519848635125783, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.75)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: -40.688412917
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': -40.68841291703659, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.69)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 2.11065173013
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 2.110651730131618, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.11)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 0.846405579662
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 0.8464055796615317, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.85)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: -10.0480312092
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': -10.048031209166215, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -10.05)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 1.02517690976
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 2, 't': 23, 'action': 'right', 'reward': 1.0251769097645862, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.03)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 0.284824371226
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 1, 't': 24, 'action': 'right', 'reward': 0.2848243712255689, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.28)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 16
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (7, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0039; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0039; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 0.782229781754
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.782229781754024, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'forward')
Agent drove right instead of left. (rewarded 0.78)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: left, reward: 2.79175665908
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 2.791756659075682, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 2.79)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: 0.760785523243
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.7607855232426269, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.76)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: right, reward: 0.195201670711
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 0.19520167071114314, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.20)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: right, reward: 1.88034244556
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.8803424455605073, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.88)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: left, reward: -39.8715605104
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': -39.87156051035683, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.87)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: right, reward: -0.00117713332645
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': -0.0011771333264458672, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded -0.00)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: left, reward: -20.5044163104
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': -20.50441631043929, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.50)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: forward, reward: 1.53367412337
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.533674123366851, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove forward instead of left. (rewarded 1.53)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: left, reward: 2.19558379568
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 2.1955837956800415, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 2.20)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 0.412554437406
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'forward'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.41255443740634246, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'forward')
Agent drove right instead of forward. (rewarded 0.41)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: -10.7823657486
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': -10.782365748645589, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent attempted driving left through a red light. (rewarded -10.78)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: -40.5824502735
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', None, 'right'), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -40.58245027348443, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.58)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 0.177770723003
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.17777072300253682, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.18)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 1.95704592856
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.9570459285580482, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.96)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: forward, reward: -10.5655031863
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -10.565503186345397, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.57)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: right, reward: 1.30115789588
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.3011578958829364, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.30)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: 2.23217821213
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 2.2321782121299214, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.23)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 0.203971593986
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 0.2039715939859017, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent drove forward instead of right. (rewarded 0.20)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: 0.476543287911
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 0.47654328791095735, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent drove forward instead of right. (rewarded 0.48)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 17
\-------------------------
Environment.reset(): Trial set up with start = (4, 3), destination = (6, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0035; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 0.273975066127
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 0.2739750661268392, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded 0.27)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 2.95388017873
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.953880178734456, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 2.95)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 1.05656314152
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.0565631415197474, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.06)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: left, reward: -9.84947911349
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': -9.849479113486236, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -9.85)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 2.47817964738
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.478179647379919, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.48)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: right, reward: 1.01288223546
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.012882235456333, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.01)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: 2.82226374417
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.822263744168127, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.82)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 1.81113152686
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.811131526861289, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.81)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: 0.445554168524
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.4455541685240797, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.45)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 0.616781712899
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.6167817128990747, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 0.62)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 2.23466050831
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.234660508314566, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.23)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 1.0850504105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.0850504105028955, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.09)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: forward, reward: 0.730189271866
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', 'left'), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 0.7301892718656497, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', 'left')
Agent drove forward instead of right. (rewarded 0.73)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 2.01889876694
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 2.0188987669422067, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.02)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 2.36506567963
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.3650656796331155, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.37)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 1.3187241273
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.3187241272959596, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.32)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: left, reward: 0.603578942804
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 0.6035789428041095, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.60)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 1.28670894487
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.2867089448696867, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.29)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 2.10328205131
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 2.1032820513058716, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.10)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: left, reward: -40.5250759447
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 6, 't': 19, 'action': 'left', 'reward': -40.52507594471372, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.53)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 1.80486581101
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.8048658110114284, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.80)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 0.850943602274
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 0.8509436022742368, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.85)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.90744645953
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 3, 't': 22, 'action': None, 'reward': 1.9074464595286909, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.91)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: -0.162554423499
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 23, 'action': 'right', 'reward': -0.16255442349877858, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded -0.16)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 0.674115903034
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': 0.6741159030335862, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent drove forward instead of right. (rewarded 0.67)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 18
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (5, 3), deadline = 30
Simulating trial. . .
epsilon = 0.0031; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: right, reward: 0.44185874207
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 0.4418587420700373, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent drove right instead of forward. (rewarded 0.44)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: right, reward: 1.3701835341
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.3701835340959163, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.37)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: -10.8661183554
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', 'left'), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': -10.866118355449302, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -10.87)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 0.930266796552
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 0.9302667965520574, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.93)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.58153304752
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.5815330475239142, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.58)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: right, reward: 1.42086856514
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.4208685651376451, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.42)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 0.981639331608
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 24, 't': 6, 'action': None, 'reward': 0.981639331607586, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.98)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: left, reward: 0.975683877379
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 0.9756838773788162, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.98)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: forward, reward: 1.78466053384
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': 1.7846605338441495, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove forward instead of right. (rewarded 1.78)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: left, reward: 1.04242972874
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 1.0424297287392126, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 1.04)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.53186465105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.5318646510504383, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.53)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 0.822685359727
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 0.8226853597269342, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.82)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: -20.6148779315
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 18, 't': 12, 'action': 'right', 'reward': -20.61487793147427, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.61)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 1.75856852987
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 1.7585685298741498, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent drove right instead of left. (rewarded 1.76)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 1.09778356289
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.0977835628935158, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.10)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 1.21798127728
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.217981277276851, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.22)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 2.01833651482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 14, 't': 16, 'action': None, 'reward': 2.018336514821307, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.02)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: 1.22308192654
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': 1.223081926541682, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent drove forward instead of left. (rewarded 1.22)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: -0.208117145832
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 12, 't': 18, 'action': 'right', 'reward': -0.20811714583245844, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded -0.21)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 1.60232959943
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 11, 't': 19, 'action': 'left', 'reward': 1.6023295994268625, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.60)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: left, reward: 0.0674464953689
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 10, 't': 20, 'action': 'left', 'reward': 0.0674464953689311, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent drove left instead of right. (rewarded 0.07)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 2.12661337822
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 9, 't': 21, 'action': None, 'reward': 2.1266133782150902, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.13)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: right, reward: -20.9100977538
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 8, 't': 22, 'action': 'right', 'reward': -20.91009775382844, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.91)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 2.27906989704
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 7, 't': 23, 'action': 'forward', 'reward': 2.2790698970388537, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.28)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: -0.403729700674
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': -0.40372970067376257, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent drove right instead of forward. (rewarded -0.40)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: -0.36327056628
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 25, 'action': 'right', 'reward': -0.36327056627964205, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded -0.36)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.48972768664
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 4, 't': 26, 'action': None, 'reward': 1.489727686639692, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.49)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.37180596112
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 3, 't': 27, 'action': None, 'reward': 1.3718059611192939, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.37)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.86093731332
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 2, 't': 28, 'action': None, 'reward': 1.8609373133165308, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.86)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: left, reward: 0.894290529369
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 1, 't': 29, 'action': 'left', 'reward': 0.894290529368863, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.89)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 19
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (3, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0028; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0028; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0028; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: forward, reward: -9.90474990479
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -9.90474990478581, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -9.90)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.72666405969
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.7266640596926401, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.73)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.46450824883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.4645082488250758, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.46)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 2.57495583699
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.5749558369873373, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.57)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 2.67750405652
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.677504056517673, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.68)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 0.973039842435
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 0.9730398424347835, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.97)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 2.15123515854
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.1512351585354255, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.15)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: forward, reward: 2.17798723206
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.1779872320639027, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.18)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 1.30774355419
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.3077435541901736, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 1.31)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 0.821625923433
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.8216259234326415, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.82)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 1.25806283053
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.2580628305300043, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.26)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 0.552395577854
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', 'left'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.5523955778537024, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', 'left')
Agent drove right instead of left. (rewarded 0.55)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: 1.11472512073
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 1.1147251207305966, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent drove forward instead of right. (rewarded 1.11)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: -5.01461965168
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 7, 't': 13, 'action': None, 'reward': -5.014619651683628, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.01)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: -4.17760587842
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'right'), 'deadline': 6, 't': 14, 'action': None, 'reward': -4.17760587841616, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.18)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 0.584679930826
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 0.5846799308255333, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 0.58)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: -0.117051766413
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': -0.11705176641326598, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded -0.12)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 0.34348619386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.3434861938595122, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.34)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 0.267559737962
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.26755973796249966, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.27)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: -0.479633041738
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -0.47963304173833887, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded -0.48)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 20
\-------------------------
Environment.reset(): Trial set up with start = (8, 7), destination = (2, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0025; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0025; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0025; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: right, reward: 0.725764595834
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.7257645958344907, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.73)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: left, reward: 1.88120136536
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.8812013653612778, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.88)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 2.76411439512
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.7641143951199147, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.76)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: left, reward: -40.7692399789
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -40.769239978892244, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.77)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: right, reward: 0.761217834164
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.7612178341642241, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 0.76)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 1.77901372589
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'right'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.7790137258918906, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'right')
Agent drove right instead of left. (rewarded 1.78)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 2.03814701928
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.0381470192827678, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.04)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: -4.49358862387
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 13, 't': 7, 'action': None, 'reward': -4.493588623870233, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.49)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 2.51198094546
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 2.511980945464157, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.51)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 0.952899235411
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.9528992354111632, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.95)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: forward, reward: 1.73142360551
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.7314236055097134, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.73)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: forward, reward: 0.0044774294469
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.004477429446899217, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.00)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: right, reward: 1.08806050208
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.08806050208332, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.09)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: left, reward: 1.36327602414
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.3632760241413724, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 1.36)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: right, reward: 0.721956325093
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 0.7219563250927291, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 0.72)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 0.142682136163
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 0.14268213616312253, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent drove forward instead of right. (rewarded 0.14)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 1.37822149023
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.3782214902270062, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.38)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 0.533329419741
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.5333294197409892, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 0.53)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 1.27317672657
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.2731767265697302, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.27)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 1.09789888556
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 1.097898885557254, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.10)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 1
\-------------------------
Environment.reset(): Trial set up with start = (3, 4), destination = (7, 2), deadline = 30
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 1.33313977451
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': 1.3331397745122886, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.33)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.40939425757
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.4093942575654161, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.41)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 0.197875853443
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 0.19787585344261205, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.20)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: 1.02357786789
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 1.023577867890586, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.02)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 1.84079388774
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 1.8407938877363867, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 1.84)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: left, reward: -39.2713948463
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': -39.271394846257934, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.27)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 2.18703271351
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 2.1870327135126773, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.19)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: -5.5640723627
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', 'right', None), 'deadline': 23, 't': 7, 'action': None, 'reward': -5.564072362696283, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.56)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 2.40721967557
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': 2.40721967557478, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 2.41)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.24149757857
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 21, 't': 9, 'action': None, 'reward': 2.2414975785681746, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.24)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.34109710971
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.3410971097052198, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.34)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.10204178461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.1020417846139807, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.10)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.33560288939
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.3356028893948875, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.34)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 1.15876462379
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.158764623788879, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.16)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 1.10871337362
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': 1.1087133736204728, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.11)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 2.63530962013
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 2.6353096201323174, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.64)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 1.31688352953
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 14, 't': 16, 'action': None, 'reward': 1.3168835295297414, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.32)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 1.86142215738
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.8614221573754794, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.86)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 2.57897943333
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 18, 'action': None, 'reward': 2.578979433331269, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.58)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 1.50217102488
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 11, 't': 19, 'action': None, 'reward': 1.5021710248788211, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.50)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 0.744185531996
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 10, 't': 20, 'action': None, 'reward': 0.7441855319963957, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.74)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: right, reward: -0.0950366272753
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'right', 'reward': -0.09503662727530171, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded -0.10)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: right, reward: -0.375452397078
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 8, 't': 22, 'action': 'right', 'reward': -0.37545239707797684, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded -0.38)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.02849280931
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 7, 't': 23, 'action': None, 'reward': 1.028492809309496, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.03)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 2.20730751348
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 6, 't': 24, 'action': None, 'reward': 2.2073075134826223, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.21)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: left, reward: 1.26645133016
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 5, 't': 25, 'action': 'left', 'reward': 1.2664513301637161, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 1.27)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: right, reward: 1.01308126717
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 4, 't': 26, 'action': 'right', 'reward': 1.0130812671694938, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.01)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: right, reward: 2.19591458808
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 27, 'action': 'right', 'reward': 2.1959145880804765, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.20)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 0.65192548746
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 2, 't': 28, 'action': None, 'reward': 0.6519254874599962, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 0.65)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 0.883400397406
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'forward'), 'deadline': 1, 't': 29, 'action': 'right', 'reward': 0.8834003974063795, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', 'left', 'forward', 'forward')
Agent drove right instead of forward. (rewarded 0.88)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 2
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (5, 5), deadline = 35
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 2.73338176968
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 35, 't': 0, 'action': None, 'reward': 2.733381769683015, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.73)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 2.97036969036
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 34, 't': 1, 'action': 'right', 'reward': 2.970369690359639, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.97)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 0.0436107618321
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 33, 't': 2, 'action': 'right', 'reward': 0.043610761832115985, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.04)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.30407273722
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 32, 't': 3, 'action': None, 'reward': 2.304072737222057, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.30)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.66595201733
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'left'), 'deadline': 31, 't': 4, 'action': None, 'reward': 2.6659520173331095, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.67)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.11150125799
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 30, 't': 5, 'action': None, 'reward': 2.1115012579920673, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.11)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: forward, reward: 1.83089200994
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 29, 't': 6, 'action': 'forward', 'reward': 1.830892009936902, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.83)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 0.360083830155
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 28, 't': 7, 'action': 'right', 'reward': 0.36008383015483336, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 0.36)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 0.322022769096
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 27, 't': 8, 'action': 'forward', 'reward': 0.3220227690962828, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 0.32)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 1.13654719025
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 26, 't': 9, 'action': None, 'reward': 1.1365471902549444, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.14)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 0.993963015631
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 25, 't': 10, 'action': None, 'reward': 0.9939630156307846, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.99)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 2.85102900521
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 24, 't': 11, 'action': 'forward', 'reward': 2.851029005208886, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.85)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.69153798618
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 12, 'action': 'right', 'reward': 1.6915379861844952, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.69)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 0.928896870582
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 13, 'action': 'right', 'reward': 0.9288968705818196, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.93)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 0.0680798237052
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 21, 't': 14, 'action': 'forward', 'reward': 0.06807982370522103, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded 0.07)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 2.02293994837
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 15, 'action': 'forward', 'reward': 2.0229399483699764, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.02)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: -40.1470616991
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'right'), 'deadline': 19, 't': 16, 'action': 'forward', 'reward': -40.147061699082776, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'right')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.15)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: -0.184262189653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 18, 't': 17, 'action': 'right', 'reward': -0.18426218965270869, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded -0.18)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: 0.948230144465
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 17, 't': 18, 'action': 'left', 'reward': 0.9482301444654759, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 0.95)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: -40.3422886053
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 16, 't': 19, 'action': 'forward', 'reward': -40.34228860529604, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.34)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 1.47184453692
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 15, 't': 20, 'action': 'right', 'reward': 1.4718445369248376, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.47)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.60201576629
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 14, 't': 21, 'action': 'right', 'reward': 1.6020157662914345, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 1.60)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.07477171849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 13, 't': 22, 'action': None, 'reward': 1.0747717184908545, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.07)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.602219044
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 12, 't': 23, 'action': None, 'reward': 1.6022190439961905, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.60)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: -9.22344624573
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'right', 'forward'), 'deadline': 11, 't': 24, 'action': 'forward', 'reward': -9.223446245731061, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.22)
29% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 1.39280046466
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 10, 't': 25, 'action': 'right', 'reward': 1.3928004646582413, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.39)
26% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: 0.64811885281
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 9, 't': 26, 'action': 'forward', 'reward': 0.648118852809906, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove forward instead of right. (rewarded 0.65)
23% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: right, reward: 2.13676197837
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 8, 't': 27, 'action': 'right', 'reward': 2.1367619783699743, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.14)
20% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: -0.112594972507
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 7, 't': 28, 'action': 'right', 'reward': -0.11259497250706518, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded -0.11)
17% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: -0.24007804925
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 6, 't': 29, 'action': 'right', 'reward': -0.24007804924998644, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent drove right instead of left. (rewarded -0.24)
14% of time remaining to reach destination.
/-------------------
| Step 30 Results
\-------------------
Environment.step(): t = 30
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: 0.483411675238
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 5, 't': 30, 'action': 'right', 'reward': 0.48341167523813566, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 0.48)
11% of time remaining to reach destination.
/-------------------
| Step 31 Results
\-------------------
Environment.step(): t = 31
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.99262833572
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 4, 't': 31, 'action': None, 'reward': 1.9926283357159909, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.99)
9% of time remaining to reach destination.
/-------------------
| Step 32 Results
\-------------------
Environment.step(): t = 32
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.43023241046
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 3, 't': 32, 'action': None, 'reward': 1.4302324104640975, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.43)
6% of time remaining to reach destination.
/-------------------
| Step 33 Results
\-------------------
Environment.step(): t = 33
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.48860713334
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 2, 't': 33, 'action': None, 'reward': 1.4886071333386683, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.49)
3% of time remaining to reach destination.
/-------------------
| Step 34 Results
\-------------------
Environment.step(): t = 34
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 0.583452133164
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 1, 't': 34, 'action': None, 'reward': 0.583452133164331, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.58)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 3
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (5, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: right, reward: 0.0442167726742
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 0.044216772674234184, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 0.04)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: right, reward: 1.69066454935
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'right'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.690664549353927, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'right')
Agent drove right instead of left. (rewarded 1.69)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 2.07087908317
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 2.0708790831749964, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.07)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.85031415474
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.8503141547371234, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.85)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.13516385765
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.135163857646054, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.14)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: forward, reward: -10.5964141271
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': -10.59641412713848, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -10.60)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: 1.09584100108
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.0958410010811965, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.10)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: 1.08031939434
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.080319394341086, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove right instead of left. (rewarded 1.08)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: -5.98588824
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', None, 'forward'), 'deadline': 17, 't': 8, 'action': None, 'reward': -5.985888239997316, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.99)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 1.15125171738
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.1512517173826968, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.15)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 2.36887170941
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'forward'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 2.368871709411452, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 2.37)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: 1.39965390561
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.3996539056097652, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.40)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 2.50449256363
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.504492563632554, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.50)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: 0.818230613858
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 0.8182306138581925, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.82)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 0.229304687955
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 0.22930468795490888, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.23)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: left, reward: 1.28380128817
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 1.2838012881665555, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent drove left instead of right. (rewarded 1.28)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 0.363792145273
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 0.3637921452725542, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove right instead of left. (rewarded 0.36)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: -4.99849025049
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 8, 't': 17, 'action': None, 'reward': -4.998490250490184, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.00)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: left, reward: -20.3774895254
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'right'), 'deadline': 7, 't': 18, 'action': 'left', 'reward': -20.37748952535473, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.38)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 0.479564352812
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 0.4795643528120117, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 0.48)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: -0.195610374382
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 5, 't': 20, 'action': 'right', 'reward': -0.19561037438230633, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent drove right instead of forward. (rewarded -0.20)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: forward, reward: 0.893667036219
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 0.8936670362194018, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.89)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: left, reward: 1.57556507101
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 3, 't': 22, 'action': 'left', 'reward': 1.5755650710083557, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 1.58)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: left, reward: 0.161698318266
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 2, 't': 23, 'action': 'left', 'reward': 0.16169831826641246, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.16)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 2.01497919926
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 1, 't': 24, 'action': 'right', 'reward': 2.014979199258314, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.01)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 4
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (5, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: -10.3985855874
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'right', 'forward'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -10.398585587448212, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'forward')
Agent attempted driving left through a red light. (rewarded -10.40)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: 2.26970881656
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.2697088165588903, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.27)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: 1.90471959043
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.904719590431487, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.90)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 0.975713947847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 0.9757139478470926, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 0.98)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.5303970755
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.5303970755021608, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.53)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.48875577009
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.4887557700853946, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.49)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 1.44308636627
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.4430863662698987, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent drove forward instead of left. (rewarded 1.44)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 0.814775196893
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.8147751968929022, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.81)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: left, reward: -39.7075604049
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'right'), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -39.707560404933005, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.71)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 0.83475184836
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.8347518483596088, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 0.83)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 0.0134709994807
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 0.01347099948071595, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.01)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 1.43080993936
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.4308099393553144, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.43)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: forward, reward: 1.21816455529
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 1.2181645552853797, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.22)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: forward, reward: 2.29542577607
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 2.2954257760721735, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.30)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 5
\-------------------------
Environment.reset(): Trial set up with start = (5, 2), destination = (1, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 1.64296397241
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.6429639724134653, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.64)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 2.666628394
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.6666283939995057, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.67)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 2.81847131027
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.818471310270057, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 2.82)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: right, reward: 1.56462893093
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.5646289309345829, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.56)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 0.798297847306
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.7982978473055732, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.80)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 1.13129876791
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.1312987679076272, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove right instead of left. (rewarded 1.13)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 1.68525866165
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.6852586616491352, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.69)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 1.57460786477
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.5746078647749737, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.57)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: right, reward: 1.37474089705
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.3747408970528274, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.37)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: 1.87950905073
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.8795090507270253, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.88)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 0.270543619413
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.2705436194132883, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.27)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 2.67234735256
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.6723473525569252, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.67)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.0912568095
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.0912568094979398, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.09)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: forward, reward: 1.25302178535
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.2530217853539845, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.25)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 1.23737460086
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.2373746008557172, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 1.24)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: left, reward: 0.928268224536
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 0.9282682245362901, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent drove left instead of right. (rewarded 0.93)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 0.301272088159
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.30127208815933915, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent drove right instead of left. (rewarded 0.30)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 0.388646336178
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.3886463361783905, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 0.39)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 0.60125923486
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.6012592348597212, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.60)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: left, reward: 0.24190290897
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 0.2419029089702921, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.24)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 6
\-------------------------
Environment.reset(): Trial set up with start = (7, 4), destination = (2, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 1.03996661395
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.039966613952289, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.04)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 2.88770207598
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.8877020759753407, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.89)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 2.96879015742
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.968790157415728, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.97)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.69284650651
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.6928465065115161, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.69)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.97411775994
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.9741177599391277, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.97)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.87457904204
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.8745790420425914, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.87)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 2.05801278647
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.058012786474737, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.06)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 0.63431506482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.6343150648196747, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.63)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: left, reward: 2.11617170328
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 2.1161717032831353, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent followed the waypoint left. (rewarded 2.12)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 1.27825770882
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.2782577088181255, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.28)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 0.103157534218
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.1031575342175185, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.10)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: -4.15328203083
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'forward'), 'deadline': 14, 't': 11, 'action': None, 'reward': -4.153282030834051, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.15)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: forward, reward: 1.56506518095
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 1.5650651809523386, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 1.57)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: forward, reward: 2.3518902896
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 2.3518902895998544, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.35)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: -0.153398109724
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -0.15339810972437695, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded -0.15)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 1.33832479475
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.3383247947543375, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.34)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 0.116393574373
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 0.11639357437311582, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove right instead of forward. (rewarded 0.12)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: -0.0128780169882
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': -0.012878016988160379, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded -0.01)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 0.99510515508
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 0.9951051550795365, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.00)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: left, reward: 0.342284806339
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 0.3422848063392835, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.34)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 0.453447260525
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.45344726052470885, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 0.45)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: left, reward: -9.41934478661
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': -9.419344786607692, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent attempted driving left through a red light. (rewarded -9.42)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: -0.688936038348
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': -0.6889360383477421, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded -0.69)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: -0.685879649144
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 23, 'action': 'right', 'reward': -0.6858796491436909, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded -0.69)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 1.58715300922
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.5871530092196215, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.59)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 7
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (6, 3), deadline = 30
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: -40.7165356095
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 30, 't': 0, 'action': 'left', 'reward': -40.71653560952479, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.72)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.57953735659
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.579537356594603, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.58)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.88848287113
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.8884828711277002, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.89)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: -39.2405927034
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 27, 't': 3, 'action': 'left', 'reward': -39.24059270343251, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.24)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.54138516947
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 1.5413851694686214, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.54)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: 1.48322754444
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 1.4832275444378467, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.48)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 0.759230073333
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 0.75923007333343, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.76)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: forward, reward: 0.957693701195
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 0.9576937011952683, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.96)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: left, reward: 1.35068807513
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': 1.3506880751331, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.35)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: right, reward: -20.8323661658
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 21, 't': 9, 'action': 'right', 'reward': -20.832366165814477, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.83)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: right, reward: 0.416796591984
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 0.41679659198426844, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.42)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: right, reward: 1.85053307261
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 1.8505330726051286, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.85)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: 1.12765078892
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.1276507889182927, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.13)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 2.45109995924
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 2.451099959242919, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.45)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: right, reward: 1.68393238341
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.683932383408504, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.68)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: left, reward: -0.151257746917
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 15, 't': 15, 'action': 'left', 'reward': -0.1512577469171209, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded -0.15)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: right, reward: 2.21149447889
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 2.2114944788947155, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.21)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.76135661203
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.7613566120269455, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.76)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.42654172769
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.4265417276933454, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.43)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.90945296148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 11, 't': 19, 'action': None, 'reward': 1.9094529614774105, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.91)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.1949697506
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 10, 't': 20, 'action': None, 'reward': 1.1949697506000252, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.19)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: left, reward: 1.15958649543
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'left', 'reward': 1.1595864954343251, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 1.16)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 2.27971427876
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 2.2797142787623845, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.28)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.43578153272
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 7, 't': 23, 'action': None, 'reward': 1.435781532723906, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.44)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.252932977
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 6, 't': 24, 'action': None, 'reward': 1.2529329769998996, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.25)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: left, reward: 0.67540095627
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 5, 't': 25, 'action': 'left', 'reward': 0.6754009562698458, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.68)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 0.802909599059
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 4, 't': 26, 'action': None, 'reward': 0.8029095990594852, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 0.80)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: left, reward: 1.54222340029
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 3, 't': 27, 'action': 'left', 'reward': 1.5422234002873114, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.54)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: forward, reward: -0.56174297491
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 2, 't': 28, 'action': 'forward', 'reward': -0.5617429749095871, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded -0.56)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: forward, reward: 0.198226804856
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 1, 't': 29, 'action': 'forward', 'reward': 0.19822680485593525, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.20)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 8
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (5, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 1.0008013307
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.0008013306993284, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.00)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 0.875395434474
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 0.8753954344736887, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent drove forward instead of left. (rewarded 0.88)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 2.92427948775
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.924279487747661, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.92)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 1.08883883611
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.0888388361057357, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.09)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: -5.17901421695
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': -5.179014216948701, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.18)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 0.141084417089
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.14108441708900343, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'forward')
Agent drove right instead of forward. (rewarded 0.14)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: 2.607268207
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 2.6072682069957933, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.61)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: -39.9211923465
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': -39.921192346505606, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.92)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.80375189283
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.803751892826, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.80)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 0.275756898348
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.2757568983484442, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.28)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: left, reward: 0.958351653214
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 0.9583516532144929, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 0.96)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 2.53027842281
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 2.5302784228090665, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.53)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 2.08279547724
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 2.0827954772412367, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.08)
48% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 9
\-------------------------
Environment.reset(): Trial set up with start = (5, 2), destination = (8, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: right, reward: 0.410543393934
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.41054339393429684, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 0.41)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: forward, reward: 1.51514093368
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.5151409336779469, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.52)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: forward, reward: -10.2701902045
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': -10.270190204451938, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent attempted driving forward through a red light. (rewarded -10.27)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 2.61452025402
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.614520254023669, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.61)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 1.81628633843
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.8162863384298447, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.82)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: right, reward: 1.17705362837
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.1770536283690474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.18)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: forward, reward: -0.0786001941136
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -0.07860019411358143, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded -0.08)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.56128979522
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.561289795223765, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 2.56)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 0.239317815039
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'right'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.23931781503910965, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'right')
Agent drove right instead of left. (rewarded 0.24)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 1.58767382238
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.5876738223778313, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 1.59)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 1.09140069471
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.0914006947142776, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.09)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 2.53771009721
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.5377100972071425, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.54)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 2.61901489502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.6190148950209524, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.62)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.37041124512
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.3704112451169401, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.37)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.58633406985
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.5863340698480763, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.59)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 2.47624871557
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.47624871557034, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.48)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: -4.52080918031
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 4, 't': 16, 'action': None, 'reward': -4.520809180311197, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.52)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: forward, reward: 1.29185754215
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 1.2918575421456624, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent drove forward instead of right. (rewarded 1.29)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: forward, reward: -9.74843678733
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': -9.748436787331899, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent attempted driving forward through a red light. (rewarded -9.75)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: left, reward: -0.368227066853
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 1, 't': 19, 'action': 'left', 'reward': -0.368227066852555, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded -0.37)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 10
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (4, 4), deadline = 30
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 1.73749710988
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.7374971098846979, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.74)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 1.80806594444
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.8080659444381637, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.81)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 2.65199308332
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.6519930833203036, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.65)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 1.7314407063
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 1.7314407063032913, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.73)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: -39.7119118322
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': -39.71191183222924, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.71)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 1.23290879909
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.232908799086988, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.23)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: -5.2872719123
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'forward'), 'deadline': 24, 't': 6, 'action': None, 'reward': -5.28727191230251, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.29)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 1.79546836477
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'forward'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.7954683647665663, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.80)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: -4.49720520535
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': -4.497205205348514, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.50)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: left, reward: 1.57407563884
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 1.574075638842976, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent followed the waypoint left. (rewarded 1.57)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.16030265758
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.160302657579072, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.16)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.35531121337
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.3553112133701577, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.36)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.56153691582
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.561536915819677, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.56)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 0.159752297517
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 0.1597522975165815, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.16)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 0.977275032429
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 0.9772750324289945, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.98)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 1.22571897091
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.2257189709135203, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.23)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: 1.25041348657
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 1.25041348656861, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.25)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.47731181326
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.4773118132611707, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.48)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 1.20712870482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 1.207128704824316, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.21)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: -39.9114112935
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': -39.91141129351916, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.91)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: -40.0784284341
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 10, 't': 20, 'action': 'forward', 'reward': -40.078428434111224, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.08)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: forward, reward: -0.0318073043885
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': -0.031807304388524194, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded -0.03)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 1.06471045385
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 1.0647104538543624, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.06)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.77000967753
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 7, 't': 23, 'action': None, 'reward': 1.770009677527949, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.77)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.1506536807
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': None, 'reward': 1.1506536806999506, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.15)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 0.886727667104
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 25, 'action': None, 'reward': 0.8867276671037179, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.89)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: left, reward: -0.13803242376
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'left'), 'deadline': 4, 't': 26, 'action': 'left', 'reward': -0.13803242375966973, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'left')
Agent drove left instead of right. (rewarded -0.14)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: left, reward: 1.87276645454
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 3, 't': 27, 'action': 'left', 'reward': 1.8727664545390732, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 1.87)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 1.52679022253
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 2, 't': 28, 'action': 'forward', 'reward': 1.52679022253446, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.53)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 0.889894447423
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 1, 't': 29, 'action': 'right', 'reward': 0.8898944474231947, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 0.89)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
Simulation ended. . .
<matplotlib.figure.Figure at 0x8848278>
In [25]:
# self.epsilon =1.0/(self.t**2)
# Load the 'sim_improved-learning' file from the improved Q-Learning simulation
vs.plot_trials('sim_improved-learning.csv')
In [26]:
%run smartcab\agent.py
/-------------------------
| Training trial 1
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (8, 6), deadline = 30
Simulating trial. . .
epsilon = 0.9900; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: -5.44574296709
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'right', 'right', None), 'deadline': 30, 't': 0, 'action': None, 'reward': -5.445742967086319, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.45)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: -5.17994212514
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': -5.179942125143327, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.18)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 1.43938774198
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 1.4393877419829946, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove right instead of left. (rewarded 1.44)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 2.75642261789
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 2.7564226178912694, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.76)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: -10.7481226909
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': 'left', 'reward': -10.748122690858883, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.75)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: -10.1307635149
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': -10.13076351486155, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.13)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 1.06007029999
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 1.0600702999885985, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.06)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 0.635950861147
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 0.6359508611468542, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove right instead of forward. (rewarded 0.64)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: forward, reward: -9.0745336395
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': -9.074533639497183, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.07)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: forward, reward: -10.157168214
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': -10.15716821398017, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -10.16)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: left, reward: 1.54365896811
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 1.54365896811375, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.54)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: -9.95695028924
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'right', 'forward'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': -9.956950289236062, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.96)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 2.09184724967
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 2.0918472496687723, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.09)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: left, reward: -39.4173236928
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'right'), 'deadline': 17, 't': 13, 'action': 'left', 'reward': -39.4173236928252, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.42)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 0.895959223331
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'right'), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 0.8959592233305453, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'right')
Agent drove right instead of forward. (rewarded 0.90)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 1.38399935223
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.3839993522265202, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 1.38)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: left, reward: -19.0589448638
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'right', None, 'right'), 'deadline': 14, 't': 16, 'action': 'left', 'reward': -19.058944863785648, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.06)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: forward, reward: 1.45617819388
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': 1.4561781938759586, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.46)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: left, reward: -10.0024996612
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 12, 't': 18, 'action': 'left', 'reward': -10.002499661163926, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent attempted driving left through a red light. (rewarded -10.00)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: left, reward: -10.0017727358
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 19, 'action': 'left', 'reward': -10.001772735756635, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.00)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 0.750065109222
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 0.7500651092218253, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 0.75)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 1.42138226564
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'left'), 'deadline': 9, 't': 21, 'action': None, 'reward': 1.421382265639358, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.42)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 0.629453202017
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 0.6294532020174002, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 0.63)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 0.126798474869
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 0.12679847486867712, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.13)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.29694229023
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 6, 't': 24, 'action': None, 'reward': 1.2969422902316516, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.30)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.24770198374
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 5, 't': 25, 'action': None, 'reward': 1.2477019837390315, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.25)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: 0.627688507782
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 4, 't': 26, 'action': 'left', 'reward': 0.6276885077820407, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 0.63)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: -9.36746241076
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 3, 't': 27, 'action': 'left', 'reward': -9.367462410760178, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent attempted driving left through a red light. (rewarded -9.37)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: -9.47718986971
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 28, 'action': 'left', 'reward': -9.47718986971059, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.48)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: forward, reward: 0.226347874097
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 1, 't': 29, 'action': 'forward', 'reward': 0.22634787409695667, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded 0.23)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 2
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (1, 3), deadline = 20
Simulating trial. . .
epsilon = 0.9802; alpha = 0.1000
Simulating trial. . .
epsilon = 0.9802; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 1.37819961637
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.3781996163694281, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.38)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.52188108445
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.5218810844544048, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.52)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: left, reward: 0.618301009203
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 0.6183010092025022, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.62)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 2.14629406991
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 2.1462940699088016, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.15)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: 0.330367512772
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 0.33036751277231036, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.33)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 1.83633744287
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.836337442873162, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.84)
70% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 3
\-------------------------
Environment.reset(): Trial set up with start = (7, 6), destination = (4, 7), deadline = 20
Simulating trial. . .
epsilon = 0.9704; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: -19.777240005
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': -19.777240005019788, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.78)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: left, reward: -40.6851797303
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -40.685179730307446, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.69)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 1.46561798324
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.4656179832427036, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.47)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 1.58837313521
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.5883731352134092, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 1.59)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: -9.73479718748
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': -9.734797187484421, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.73)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: 1.88113360103
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.8811336010268072, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 1.88)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: -39.8056545034
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -39.80565450337667, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.81)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: -9.97781957847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -9.977819578465146, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.98)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: -10.1717293254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -10.171729325352105, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.17)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.1064991594
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.1064991593996956, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.11)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: -39.3888463022
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': -39.388846302210254, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.39)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: forward, reward: 1.07721370021
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.0772137002103361, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.08)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 1.57892331796
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.5789233179593611, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.58)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 0.946457213776
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 0.9464572137762401, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.95)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 1.34850599557
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.3485059955684902, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.35)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: forward, reward: -10.6780535882
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -10.678053588208606, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.68)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: left, reward: -19.7438617613
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 4, 't': 16, 'action': 'left', 'reward': -19.743861761282975, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.74)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: -5.4196231203
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 3, 't': 17, 'action': None, 'reward': -5.419623120296322, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.42)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: -0.181619916257
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': -0.18161991625744878, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded -0.18)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: -5.94778328403
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 1, 't': 19, 'action': None, 'reward': -5.947783284026022, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.95)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 4
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (7, 5), deadline = 30
Simulating trial. . .
epsilon = 0.9608; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: forward, reward: -40.4790843695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': -40.47908436952858, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.48)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: forward, reward: -10.7219802889
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': -10.721980288928618, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.72)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: forward, reward: -10.0390833786
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': -10.039083378599246, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.04)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: forward, reward: -39.5836942928
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', 'left'), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': -39.58369429276206, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.58)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: 2.82065620002
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 2.8206562000166153, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.82)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 1.25178117644
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.2517811764392195, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.25)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.02414885791
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.0241488579122837, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.02)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 2.33202775279
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 2.3320277527871207, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 2.33)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: left, reward: 0.748017988211
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': 0.7480179882106562, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove left instead of forward. (rewarded 0.75)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: left, reward: -20.2312695974
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 21, 't': 9, 'action': 'left', 'reward': -20.231269597361788, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.23)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: left, reward: 0.235009918699
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 0.2350099186992649, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 0.24)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: -5.59441408451
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': None, 'reward': -5.594414084507411, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.59)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.58386003399
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.5838600339889966, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.58)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: -4.8397374668
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': None, 'reward': -4.839737466803528, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.84)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: left, reward: 1.74515967996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 14, 'action': 'left', 'reward': 1.745159679959719, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 1.75)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 0.144674137291
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 0.1446741372905166, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 0.14)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: 0.356466160248
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 0.35646616024826616, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent drove left instead of right. (rewarded 0.36)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: -20.6935859117
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 13, 't': 17, 'action': 'right', 'reward': -20.693585911720366, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.69)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: -10.0070936817
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'left'), 'deadline': 12, 't': 18, 'action': 'left', 'reward': -10.00709368167731, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'left')
Agent attempted driving left through a red light. (rewarded -10.01)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: -10.8146139827
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 11, 't': 19, 'action': 'left', 'reward': -10.814613982702815, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.81)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: forward, reward: -10.7964633104
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 20, 'action': 'forward', 'reward': -10.796463310391218, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.80)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: 0.461231144671
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': None, 'reward': 0.4612311446706593, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.46)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: -0.471198189173
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': -0.4711981891731458, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent drove forward instead of left. (rewarded -0.47)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 0.540165410236
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 0.5401654102363128, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 0.54)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: -0.114577523445
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 6, 't': 24, 'action': 'right', 'reward': -0.11457752344478211, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent drove right instead of left. (rewarded -0.11)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 1.11393297157
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 5, 't': 25, 'action': None, 'reward': 1.1139329715680073, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.11)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 1.00692170385
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', None), 'deadline': 4, 't': 26, 'action': None, 'reward': 1.0069217038470428, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 1.01)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: left, reward: -19.4364929737
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 3, 't': 27, 'action': 'left', 'reward': -19.436492973670628, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.44)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: left, reward: -0.756567062181
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 2, 't': 28, 'action': 'left', 'reward': -0.7565670621806276, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent drove left instead of right. (rewarded -0.76)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: right, reward: -0.0183751975421
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 1, 't': 29, 'action': 'right', 'reward': -0.018375197542095156, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent drove right instead of forward. (rewarded -0.02)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 5
\-------------------------
Environment.reset(): Trial set up with start = (5, 2), destination = (2, 7), deadline = 20
Simulating trial. . .
epsilon = 0.9512; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: left, reward: 0.621699372202
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 0.6216993722022481, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove left instead of right. (rewarded 0.62)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: left, reward: -10.8729335983
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -10.872933598315834, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent attempted driving left through a red light. (rewarded -10.87)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: -10.5032626123
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': -10.503262612348756, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.50)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.08023043285
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.0802304328451, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.08)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 0.353299411635
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.3532994116350524, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.35)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: -4.55194414075
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': -4.551944140745095, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.55)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: -4.76883246108
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'right', 'right'), 'deadline': 14, 't': 6, 'action': None, 'reward': -4.768832461082224, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.77)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.61885536122
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.618855361216396, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.62)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: left, reward: -9.64028413232
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -9.640284132323469, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.64)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.3891516126
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.3891516125985408, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.39)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: left, reward: -19.3404188429
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'forward'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -19.34041884286633, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.34)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: -5.18303576872
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 9, 't': 11, 'action': None, 'reward': -5.183035768717453, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.18)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: right, reward: 0.519481080731
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.519481080730624, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 0.52)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: 1.03969714572
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.03969714572276, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.04)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 0.213806304837
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 0.21380630483721685, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.21)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: left, reward: -9.78931838679
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -9.78931838678675, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.79)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: -0.0377493056631
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': None, 'reward': -0.03774930566306933, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.04)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: -0.655268494265
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': -0.6552684942647057, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded -0.66)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: 1.15871072078
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 1.1587107207765581, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.16)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: left, reward: -19.841542098
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, 'right', 'right'), 'deadline': 1, 't': 19, 'action': 'left', 'reward': -19.841542098003398, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, 'right', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.84)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 6
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (3, 3), deadline = 20
Simulating trial. . .
epsilon = 0.9418; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 0.496043637237
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 0.4960436372372736, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.50)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 0.609643805337
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.6096438053373482, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.61)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 2.68229181359
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.6822918135877107, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.68)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: forward, reward: 1.67107880977
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.6710788097736287, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 1.67)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: right, reward: 2.66595937148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.665959371478775, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 2.67)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: left, reward: -10.7663778146
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': -10.766377814585852, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.77)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: -9.8291175123
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -9.829117512299202, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.83)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 0.546119398951
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.5461193989509456, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.55)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: forward, reward: -10.5368224644
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -10.536822464424269, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.54)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: right, reward: 0.0778860888985
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.07788608889848492, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.08)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: -4.68377287456
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': -4.683772874557804, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.68)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 0.486251102435
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 0.48625110243465297, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.49)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: right, reward: 1.99808412834
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.998084128340084, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.00)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: forward, reward: 0.621488021636
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.6214880216364306, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent drove forward instead of right. (rewarded 0.62)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 2.51062923464
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.5106292346386914, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.51)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 1.02814685549
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.0281468554873665, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.03)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: left, reward: -0.185877202161
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': -0.1858772021608046, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded -0.19)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 1.43757111046
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 1.4375711104580289, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.44)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: left, reward: -0.0545588727
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -0.05455887270003523, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove left instead of right. (rewarded -0.05)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: forward, reward: -9.11316491552
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -9.113164915518185, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -9.11)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 7
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (6, 4), deadline = 25
Simulating trial. . .
epsilon = 0.9324; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: None, reward: 2.19600451546
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.1960045154638532, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', 'left')
Agent properly idled at a red light. (rewarded 2.20)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 1.16825710979
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.168257109788944, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.17)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: -5.4523962873
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': -5.452396287301015, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.45)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 1.4747962339
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.4747962338960743, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 1.47)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 1.413367386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.4133673859975613, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.41)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: -10.3723440893
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': -10.372344089296218, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.37)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 1.89807796434
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.8980779643367733, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.90)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: -5.23018904472
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 18, 't': 7, 'action': None, 'reward': -5.23018904471676, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.23)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: forward, reward: 1.21856448946
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 1.2185644894649323, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.22)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: -5.07466531971
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 16, 't': 9, 'action': None, 'reward': -5.074665319709089, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.07)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: forward, reward: 1.705696183
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 1.7056961830002224, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.71)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: None, reward: -5.67352664688
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 14, 't': 11, 'action': None, 'reward': -5.6735266468756205, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.67)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: right, reward: 0.0231611116039
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 0.023161111603912987, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded 0.02)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: left, reward: 1.31723174633
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 1.3172317463326786, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 1.32)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: -0.0959897603337
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'right', 'left'), 'deadline': 11, 't': 14, 'action': None, 'reward': -0.09598976033372153, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.10)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: -4.82010012659
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 10, 't': 15, 'action': None, 'reward': -4.8201001265851, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.82)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 0.177629093094
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'right'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 0.1776290930941017, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'right')
Agent drove right instead of left. (rewarded 0.18)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: 0.372191736204
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 0.3721917362039695, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove forward instead of right. (rewarded 0.37)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: right, reward: 1.55780829443
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 1.5578082944337022, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.56)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 1.26129685669
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 1.2612968566857305, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent drove right instead of left. (rewarded 1.26)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: None, reward: 1.99112441418
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.991124414178817, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.99)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: None, reward: -4.33631496312
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 4, 't': 21, 'action': None, 'reward': -4.336314963115969, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.34)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: left, reward: 0.857621382448
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', None), 'deadline': 3, 't': 22, 'action': 'left', 'reward': 0.8576213824478705, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', None)
Agent drove left instead of right. (rewarded 0.86)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: -0.0567262352543
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 2, 't': 23, 'action': 'forward', 'reward': -0.05672623525425524, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded -0.06)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: 0.373907296383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': 0.3739072963831933, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 0.37)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 8
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (4, 6), deadline = 25
Simulating trial. . .
epsilon = 0.9231; alpha = 0.1000
Simulating trial. . .
epsilon = 0.9231; alpha = 0.1000
Simulating trial. . .
epsilon = 0.9231; alpha = 0.1000
Simulating trial. . .
epsilon = 0.9231; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 0.766267487594
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 0.7662674875942876, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.77)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: -5.69909618543
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': -5.6990961854306414, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.70)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 0.180009947144
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 0.1800099471436396, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.18)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: -9.65215330049
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': -9.652153300493449, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -9.65)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 2.04488827978
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.044888279783067, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.04)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 0.316498073532
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.3164980735320976, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.32)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: forward, reward: -9.04316435187
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -9.043164351871042, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.04)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.26941979403
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.2694197940257619, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent drove right instead of left. (rewarded 1.27)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: -4.33872415775
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 17, 't': 8, 'action': None, 'reward': -4.338724157753456, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.34)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: left, reward: 1.27206636383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.2720663638325762, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 1.27)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: None, reward: -4.472698536
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 15, 't': 10, 'action': None, 'reward': -4.4726985359997515, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.47)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: left, reward: -40.3043616167
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': -40.304361616671656, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.30)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: left, reward: -10.5312667345
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 13, 't': 12, 'action': 'left', 'reward': -10.53126673450728, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent attempted driving left through a red light. (rewarded -10.53)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: left, reward: 0.482690110832
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 0.4826901108318269, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.48)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: left, reward: -10.756411476
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -10.756411476038728, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.76)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 1.5552328698
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.5552328698037539, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.56)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: forward, reward: 0.210754592549
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 0.21075459254890416, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove forward instead of left. (rewarded 0.21)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: forward, reward: -39.799647464
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': -39.799647463972946, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.80)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 1.73098842605
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.7309884260533575, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.73)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: 2.09970809598
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 2.099708095984411, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 2.10)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: -5.49772168805
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 5, 't': 20, 'action': None, 'reward': -5.497721688047113, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.50)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: -39.5298937843
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': -39.52989378425552, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.53)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 0.960816687356
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.960816687355647, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.96)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: forward, reward: -0.387081036454
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 2, 't': 23, 'action': 'forward', 'reward': -0.38708103645406866, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded -0.39)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: None, reward: 1.77946425208
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.7794642520820367, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.78)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 9
\-------------------------
Environment.reset(): Trial set up with start = (2, 7), destination = (5, 3), deadline = 25
Simulating trial. . .
epsilon = 0.9139; alpha = 0.1000
Simulating trial. . .
epsilon = 0.9139; alpha = 0.1000
Simulating trial. . .
epsilon = 0.9139; alpha = 0.1000
Simulating trial. . .
epsilon = 0.9139; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: -10.1569298458
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -10.156929845791673, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -10.16)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: -39.6781460999
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'left'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -39.6781460998833, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.68)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.87278876784
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.8727887678388957, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.87)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: -10.408089897
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': -10.408089896961588, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.41)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 1.80466000018
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.8046600001808812, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.80)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: 1.1912772444
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.1912772444049962, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.19)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: -9.9760311964
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -9.976031196395708, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent attempted driving forward through a red light. (rewarded -9.98)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: left, reward: 1.62391078818
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 1.6239107881780328, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.62)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: -4.70624535494
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': -4.7062453549434835, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.71)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: 2.66103307864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.6610330786426077, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.66)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: -0.0130199438327
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': -0.013019943832653813, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove forward instead of right. (rewarded -0.01)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: left, reward: -40.1236810942
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': -40.12368109415518, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.12)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.62245603626
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.622456036263967, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.62)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 2.17557245695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 2.1755724569524695, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.18)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: left, reward: -10.0569334234
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -10.056933423372161, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.06)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: forward, reward: -9.77277051848
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': -9.772770518475056, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.77)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: forward, reward: 0.0739762067748
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'forward'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 0.07397620677475947, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'forward')
Agent drove forward instead of right. (rewarded 0.07)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: forward, reward: -10.7092944208
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': -10.70929442077694, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -10.71)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: forward, reward: -9.80432968243
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': -9.804329682425088, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.80)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: forward, reward: -0.047693059025
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': -0.04769305902496612, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded -0.05)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: left, reward: -39.5012931953
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 5, 't': 20, 'action': 'left', 'reward': -39.50129319532159, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.50)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 0.36561327816
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 0.3656132781599928, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 0.37)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 0.800268112268
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.8002681122684221, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 0.80)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: left, reward: -9.88531578063
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 23, 'action': 'left', 'reward': -9.885315780633029, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.89)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: forward, reward: -9.78593931484
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': -9.785939314835733, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -9.79)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 10
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (7, 6), deadline = 25
Simulating trial. . .
epsilon = 0.9048; alpha = 0.1000
Simulating trial. . .
epsilon = 0.9048; alpha = 0.1000
Simulating trial. . .
epsilon = 0.9048; alpha = 0.1000
Simulating trial. . .
epsilon = 0.9048; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 1.06219786628
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.0621978662763967, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.06)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.09226518371
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.092265183707057, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.09)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: -10.2280964754
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': 'left', 'reward': -10.228096475363774, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.23)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: -9.2183539384
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': -9.218353938400142, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.22)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: -10.3628098531
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -10.362809853111164, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.36)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: left, reward: 1.78641479286
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.7864147928592753, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 1.79)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: left, reward: -39.3735947338
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'right'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': -39.373594733786774, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.37)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: 1.05951241778
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 1.0595124177839592, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 1.06)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: -10.7176158076
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -10.717615807615758, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.72)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: 0.780260921098
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.7802609210977633, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.78)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: -4.77436972094
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 15, 't': 10, 'action': None, 'reward': -4.774369720939347, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.77)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: -40.4665523516
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': -40.46655235162297, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.47)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 1.82180718943
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.821807189432148, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.82)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 1.46094979416
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.4609497941561707, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.46)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 2.14580064437
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'right'), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.145800644373954, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'right')
Agent properly idled at a red light. (rewarded 2.15)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 2.22796419674
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'right'), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.227964196736128, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'right')
Agent properly idled at a red light. (rewarded 2.23)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: forward, reward: 1.51762665369
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 1.517626653691444, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.52)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: 1.9495390677
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 1.9495390677003042, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 1.95)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: -4.87648128189
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': -4.8764812818933345, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.88)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 0.894944915826
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 0.8949449158260667, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 0.89)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 2.1353717649
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 5, 't': 20, 'action': 'forward', 'reward': 2.1353717648971227, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.14)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: left, reward: -9.54440312396
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': -9.544403123957444, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.54)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 0.389586158865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 3, 't': 22, 'action': None, 'reward': 0.389586158864784, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.39)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: left, reward: 0.736819983038
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 2, 't': 23, 'action': 'left', 'reward': 0.7368199830381614, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.74)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: forward, reward: -0.700980865282
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': -0.7009808652820362, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent drove forward instead of right. (rewarded -0.70)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 11
\-------------------------
Environment.reset(): Trial set up with start = (7, 3), destination = (2, 5), deadline = 25
Simulating trial. . .
epsilon = 0.8958; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 0.147804111283
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 0.14780411128274373, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.15)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: 1.48002765697
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 1.480027656968246, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.48)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: left, reward: -10.0662045201
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': -10.066204520092768, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', None)
Agent attempted driving left through a red light. (rewarded -10.07)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 2.94205142818
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.942051428180611, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.94)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 2.53601119871
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.5360111987085565, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.54)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 1.56154730008
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.5615473000827358, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.56)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 0.548805781912
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', 'right'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.5488057819119657, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', 'right')
Agent drove right instead of forward. (rewarded 0.55)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: left, reward: 2.49484539515
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 2.494845395150148, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 2.49)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: left, reward: -19.2931502712
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 17, 't': 8, 'action': 'left', 'reward': -19.293150271175982, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.29)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: right, reward: 0.0794575921659
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.07945759216588466, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.08)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 0.93068841383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'left', None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 0.9306884138299166, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'left', None)
Agent drove forward instead of right. (rewarded 0.93)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 1.49864279238
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'right'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.4986427923808519, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'right')
Agent followed the waypoint right. (rewarded 1.50)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: left, reward: 1.50831149258
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 12, 'action': 'left', 'reward': 1.5083114925793435, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 1.51)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: -40.1252356369
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': -40.12523563685564, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.13)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: -40.8466550625
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': -40.84665506246421, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.85)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 1.42489968275
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.4248996827480513, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.42)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: forward, reward: -0.295951265107
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': -0.2959512651067563, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove forward instead of left. (rewarded -0.30)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: 1.09784091366
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 1.097840913663643, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.10)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: left, reward: -40.3279784727
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 7, 't': 18, 'action': 'left', 'reward': -40.32797847266583, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.33)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: -10.1991205225
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': -10.199120522466266, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.20)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: left, reward: -10.9888210766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 5, 't': 20, 'action': 'left', 'reward': -10.98882107660726, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent attempted driving left through a red light. (rewarded -10.99)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: 1.27192461925
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 1.2719246192499563, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.27)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 0.325046517338
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 22, 'action': None, 'reward': 0.3250465173379886, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.33)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: right, reward: 0.363561854941
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 2, 't': 23, 'action': 'right', 'reward': 0.3635618549414561, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.36)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: -0.15341530077
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 1, 't': 24, 'action': 'right', 'reward': -0.15341530076992438, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent drove right instead of left. (rewarded -0.15)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 12
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (1, 3), deadline = 30
Simulating trial. . .
epsilon = 0.8869; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: right, reward: 1.04606254757
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.0460625475694565, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded 1.05)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 1.48864085725
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.4886408572490462, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.49)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.36803259444
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.3680325944421288, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.37)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: left, reward: -10.3761422606
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'left', 'reward': -10.376142260614936, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.38)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: right, reward: 1.20568707059
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 1.2056870705931497, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 1.21)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: left, reward: -10.3304390777
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': -10.330439077688586, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent attempted driving left through a red light. (rewarded -10.33)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: None, reward: 2.26594342427
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.2659434242676832, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.27)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: forward, reward: -0.0358448394674
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': -0.03584483946743411, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded -0.04)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: -4.59844641595
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 22, 't': 8, 'action': None, 'reward': -4.598446415954443, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.60)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: 1.11753006534
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 1.1175300653413827, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.12)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 2.06673945585
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 2.0667394558525762, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.07)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: -10.7014313527
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': -10.701431352670634, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.70)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: -10.1465676184
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', 'left'), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': -10.146567618443303, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', 'left')
Agent attempted driving forward through a red light. (rewarded -10.15)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: -0.0335365855139
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': -0.03353658551391181, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent drove right instead of forward. (rewarded -0.03)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: forward, reward: 0.646632597607
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': 0.6466325976070868, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove forward instead of left. (rewarded 0.65)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 1.28756183237
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.287561832373425, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.29)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: forward, reward: -39.5097655597
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': -39.509765559749695, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.51)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: 1.83169225131
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 17, 'action': 'left', 'reward': 1.8316922513147356, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.83)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: -10.2341221117
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 12, 't': 18, 'action': 'forward', 'reward': -10.234122111666773, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent attempted driving forward through a red light. (rewarded -10.23)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: -4.67307959408
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 19, 'action': None, 'reward': -4.6730795940762295, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.67)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: left, reward: 0.434056601116
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 20, 'action': 'left', 'reward': 0.4340566011160609, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.43)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: -5.65205453134
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 9, 't': 21, 'action': None, 'reward': -5.652054531339181, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.65)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: left, reward: 0.890798311672
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', None), 'deadline': 8, 't': 22, 'action': 'left', 'reward': 0.890798311672452, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', None)
Agent drove left instead of right. (rewarded 0.89)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: -39.9720073806
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 7, 't': 23, 'action': 'forward', 'reward': -39.97200738057972, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.97)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: -10.949408507
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': 'forward', 'reward': -10.949408506951348, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.95)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 1.06928413252
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 1.0692841325238853, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.07)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: left, reward: -40.088672835
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'forward'), 'deadline': 4, 't': 26, 'action': 'left', 'reward': -40.088672835032234, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.09)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: forward, reward: -10.1034072752
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 3, 't': 27, 'action': 'forward', 'reward': -10.10340727515219, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.10)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 1.90715675113
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 2, 't': 28, 'action': 'right', 'reward': 1.9071567511311125, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.91)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: forward, reward: -39.9628078081
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 1, 't': 29, 'action': 'forward', 'reward': -39.962807808069115, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.96)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 13
\-------------------------
Environment.reset(): Trial set up with start = (4, 3), destination = (2, 5), deadline = 20
Simulating trial. . .
epsilon = 0.8781; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: 0.124242888737
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 0.12424288873704858, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'left', 'left')
Agent drove forward instead of right. (rewarded 0.12)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: right, reward: 2.05859710694
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.058597106943517, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'left')
Agent followed the waypoint right. (rewarded 2.06)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: right, reward: 2.42245383377
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.4224538337694623, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.42)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 0.482389260224
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'right'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 0.48238926022362516, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'right')
Agent drove right instead of forward. (rewarded 0.48)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: forward, reward: -10.3416341198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': -10.341634119799041, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.34)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 0.14122906419
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 0.14122906419002734, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.14)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 1.84787983066
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.8478798306601611, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.85)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: left, reward: 2.41609754787
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 2.416097547874471, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.42)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: left, reward: 1.63590506654
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.6359050665389243, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.64)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 1.61111032423
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.611110324229024, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.61)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: -39.2932165061
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'right'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -39.29321650612559, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.29)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 0.694181937986
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.6941819379859342, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove right instead of left. (rewarded 0.69)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: left, reward: 0.538908628374
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 0.5389086283743416, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove left instead of right. (rewarded 0.54)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 1.5050213533
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.5050213532995453, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent drove forward instead of left. (rewarded 1.51)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 1.50523889409
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.50523889408871, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.51)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: -5.0495465379
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': -5.049546537897099, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.05)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: -0.472963625734
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': -0.4729636257341656, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove right instead of left. (rewarded -0.47)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: -4.7406480717
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 3, 't': 17, 'action': None, 'reward': -4.74064807169559, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.74)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: left, reward: 0.208789294761
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': 0.20878929476075414, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove left instead of right. (rewarded 0.21)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 0.464682260916
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'right'), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.4646822609158239, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 0.46)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 14
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (6, 2), deadline = 25
Simulating trial. . .
epsilon = 0.8694; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: -9.80994720107
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', 'right', None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -9.809947201070692, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', None)
Agent attempted driving forward through a red light. (rewarded -9.81)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: -9.05400058117
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': -9.054000581168921, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.05)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.28688395785
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.2868839578489935, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.29)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: -9.59237103826
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': -9.592371038257507, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.59)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: -10.4941450344
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'right'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -10.494145034382585, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'right')
Agent attempted driving forward through a red light. (rewarded -10.49)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.73506517891
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.7350651789129494, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.74)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 1.0725475793
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.0725475792976873, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent drove forward instead of right. (rewarded 1.07)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: 0.0218865291647
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 0.021886529164691026, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 0.02)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: -10.3545222921
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -10.354522292139963, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.35)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 0.282477920443
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.2824779204426604, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent drove right instead of left. (rewarded 0.28)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: 0.637427096901
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'left'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 0.637427096901256, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'left')
Agent drove forward instead of left. (rewarded 0.64)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 2.43215939857
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.4321593985709558, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.43)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 1.2818300414
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.2818300414033945, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 1.28)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: left, reward: -9.38612569482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': -9.386125694824466, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.39)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: forward, reward: -9.09343808574
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': -9.09343808574097, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.09)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: left, reward: -9.7875996031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 15, 'action': 'left', 'reward': -9.787599603100329, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.79)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 1.29658983983
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 1.296589839832572, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 1.30)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: -19.2383206952
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': -19.238320695229536, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.24)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: -10.6283422031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 7, 't': 18, 'action': 'left', 'reward': -10.628342203119228, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.63)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: -9.39464311401
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': -9.394643114012233, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.39)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 0.886302120181
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.8863021201810921, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 0.89)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: left, reward: 0.140599539341
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 0.1405995393413454, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 0.14)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: -5.68321322389
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 3, 't': 22, 'action': None, 'reward': -5.683213223885256, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.68)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: forward, reward: 1.07691453488
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 2, 't': 23, 'action': 'forward', 'reward': 1.0769145348764202, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove forward instead of right. (rewarded 1.08)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: forward, reward: -10.2759035114
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': -10.2759035114223, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.28)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 15
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (8, 2), deadline = 30
Simulating trial. . .
epsilon = 0.8607; alpha = 0.1000
Simulating trial. . .
epsilon = 0.8607; alpha = 0.1000
Simulating trial. . .
epsilon = 0.8607; alpha = 0.1000
Simulating trial. . .
epsilon = 0.8607; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: -9.36567613507
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': -9.365676135065794, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.37)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: -10.5382658512
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': -10.538265851163445, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -10.54)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: 0.338061353779
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 0.3380613537791769, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 0.34)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 1.2456678526
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.2456678526032692, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.25)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: left, reward: 0.345622891959
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 0.3456228919593799, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 0.35)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: forward, reward: 0.155514133222
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 0.1555141332219231, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.16)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: left, reward: 1.33677092175
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 1.3367709217488544, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.34)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: left, reward: 1.45832608293
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 1.4583260829331417, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.46)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 2.85152692716
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 2.8515269271595054, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent followed the waypoint right. (rewarded 2.85)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 1.62779065449
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.6277906544892171, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.63)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: left, reward: 0.941930525914
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 0.9419305259141952, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent drove left instead of forward. (rewarded 0.94)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: left, reward: -40.843651001
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': -40.84365100103995, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.84)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: None, reward: 1.59490810512
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.5949081051245788, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.59)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: forward, reward: 0.342056205271
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 0.34205620527134917, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent drove forward instead of right. (rewarded 0.34)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: -20.3296802786
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': -20.32968027862733, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.33)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: forward, reward: -10.1413858828
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': -10.141385882830262, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.14)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 1.82595134985
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 1.8259513498526185, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.83)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 0.594438652319
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 13, 't': 17, 'action': None, 'reward': 0.5944386523188487, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.59)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 0.485258075659
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 12, 't': 18, 'action': 'forward', 'reward': 0.4852580756589766, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 0.49)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 0.641033670062
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 0.6410336700618148, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove right instead of left. (rewarded 0.64)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 1.34908585921
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 20, 'action': None, 'reward': 1.3490858592086574, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.35)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.10385963712
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 1.1038596371160894, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.10)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.69409619096
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 22, 'action': None, 'reward': 1.694096190959762, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.69)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 1.97065456103
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 1.9706545610314405, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.97)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: -0.219726659245
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': -0.21972665924493961, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded -0.22)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: -0.364566023552
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 5, 't': 25, 'action': 'forward', 'reward': -0.3645660235523287, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded -0.36)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: forward, reward: 0.109123068857
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 4, 't': 26, 'action': 'forward', 'reward': 0.10912306885712342, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 0.11)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: left, reward: -20.246905635
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 3, 't': 27, 'action': 'left', 'reward': -20.246905634958402, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.25)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: right, reward: -0.321852184964
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 2, 't': 28, 'action': 'right', 'reward': -0.3218521849635082, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove right instead of left. (rewarded -0.32)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: None, reward: -5.98672620444
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 1, 't': 29, 'action': None, 'reward': -5.986726204437035, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.99)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 16
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (7, 4), deadline = 20
Simulating trial. . .
epsilon = 0.8521; alpha = 0.1000
Simulating trial. . .
epsilon = 0.8521; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: None, reward: 1.5588337623
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.5588337622957773, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.56)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 2.88852877358
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.888528773579731, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.89)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.95328867586
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.9532886758649322, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.95)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 2.83669791564
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 2.836697915640392, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.84)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: -10.1690788823
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': -10.169078882327982, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.17)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: forward, reward: 1.74051860198
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.740518601975559, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.74)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: -4.24816089566
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': -4.248160895658864, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.25)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: -4.32852373039
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'right'), 'deadline': 13, 't': 7, 'action': None, 'reward': -4.32852373038926, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.33)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 0.720058954044
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.7200589540435834, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 0.72)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: left, reward: 0.838743725157
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 0.8387437251574685, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 0.84)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 0.112081996647
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.11208199664678398, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.11)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: 1.00380585996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.003805859955985, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.00)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 0.943800419273
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.9438004192733458, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.94)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 2.29371185952
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 2.2937118595157022, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.29)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: left, reward: -10.9170175958
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -10.917017595779047, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.92)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 2.22382468195
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.223824681946007, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.22)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: forward, reward: -9.87418511556
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': -9.874185115559023, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.87)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: forward, reward: -0.412974962813
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -0.4129749628127134, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove forward instead of right. (rewarded -0.41)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: -0.767672762512
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': -0.7676727625123856, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.77)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 1.02326677313
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.0232667731330523, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.02)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 17
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (6, 3), deadline = 25
Simulating trial. . .
epsilon = 0.8437; alpha = 0.1000
Simulating trial. . .
epsilon = 0.8437; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: -10.9387561381
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', 'right', 'forward'), 'deadline': 25, 't': 0, 'action': 'left', 'reward': -10.938756138067932, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', 'forward')
Agent attempted driving left through a red light. (rewarded -10.94)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: 2.26206135864
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.262061358635801, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.26)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: forward, reward: 2.8071527876
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 2.8071527876010363, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.81)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: left, reward: -9.26142677085
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': -9.261426770847603, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.26)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: right, reward: 0.159910311813
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 0.15991031181301218, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.16)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 2.03477659808
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.034776598083688, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.03)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 1.91211554627
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'right'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.9121155462665447, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 1.91)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: left, reward: -9.57972472143
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 18, 't': 7, 'action': 'left', 'reward': -9.579724721427732, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent attempted driving left through a red light. (rewarded -9.58)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: forward, reward: -9.80171889898
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -9.801718898981996, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.80)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: left, reward: 1.19470861065
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.194708610647703, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.19)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: -39.150851548
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': -39.15085154801509, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.15)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 2.07190693746
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 2.071906937461254, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.07)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 1.13528083173
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', 'left'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.1352808317253595, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', 'left')
Agent drove right instead of forward. (rewarded 1.14)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 0.887444605239
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 12, 't': 13, 'action': None, 'reward': 0.8874446052392899, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 0.89)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: right, reward: 1.44073068507
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 1.440730685067491, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 1.44)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 1.81968722184
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.8196872218389488, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.82)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: -5.58437868111
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': -5.584378681106424, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.58)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: -10.05590724
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': -10.055907240008146, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.06)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 0.568732241439
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 0.568732241438969, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.57)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: -5.38538912067
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 6, 't': 19, 'action': None, 'reward': -5.385389120672281, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.39)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: -4.85668114911
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': -4.85668114911387, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.86)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 1.70668679971
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 1.7066867997135868, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.71)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: forward, reward: 0.77458644492
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': 0.7745864449197806, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 0.77)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: -0.789607176549
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', 'forward'), 'deadline': 2, 't': 23, 'action': 'right', 'reward': -0.7896071765489554, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', 'forward')
Agent drove right instead of forward. (rewarded -0.79)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: -5.24479015407
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': -5.244790154065118, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.24)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 18
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (5, 2), deadline = 20
Simulating trial. . .
epsilon = 0.8353; alpha = 0.1000
Simulating trial. . .
epsilon = 0.8353; alpha = 0.1000
Simulating trial. . .
epsilon = 0.8353; alpha = 0.1000
Simulating trial. . .
epsilon = 0.8353; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: left, reward: 1.05088034091
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.0508803409050158, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove left instead of right. (rewarded 1.05)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: left, reward: -9.12636856145
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -9.126368561449226, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.13)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 0.194239909953
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.1942399099530655, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent drove right instead of left. (rewarded 0.19)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: 2.78061430539
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.780614305385286, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.78)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.26117386122
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.2611738612166576, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.26)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: left, reward: -10.6167166865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': -10.616716686462437, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.62)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: left, reward: 1.48597250053
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.4859725005256577, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 1.49)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: 1.52445819282
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.5244581928156486, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent drove forward instead of right. (rewarded 1.52)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: 1.63853641373
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.638536413732975, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 1.64)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 0.379992497133
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.3799924971332347, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded 0.38)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: right, reward: 1.31430246119
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.314302461188103, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 1.31)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: forward, reward: 0.221731855235
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.22173185523505023, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.22)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: forward, reward: -9.83074684249
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': -9.830746842492925, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.83)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: left, reward: 0.716438750084
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.7164387500837877, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.72)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: forward, reward: -0.330469908788
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': -0.330469908787833, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove forward instead of left. (rewarded -0.33)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 1.91609077624
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.9160907762393917, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.92)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 0.852368667199
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.8523686671985922, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.85)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: 0.162669696891
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.1626696968909095, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.16)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: -10.7828739347
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': -10.78287393472451, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.78)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: -10.6514364406
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -10.651436440603463, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.65)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 19
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (5, 7), deadline = 20
Simulating trial. . .
epsilon = 0.8270; alpha = 0.1000
Simulating trial. . .
epsilon = 0.8270; alpha = 0.1000
Simulating trial. . .
epsilon = 0.8270; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 0.163120132615
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.16312013261478908, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.16)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 0.438881683218
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.43888168321789356, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 0.44)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 2.25854895823
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.2585489582283342, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.26)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: -5.85547641808
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': -5.855476418082224, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.86)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 1.36438630692
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.3643863069210413, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.36)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 1.90573927883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.9057392788268863, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.91)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: -4.27977560477
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': -4.279775604774191, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.28)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: -9.69596889534
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -9.69596889534336, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.70)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 1.81227974417
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.8122797441671858, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.81)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: left, reward: -20.4944117481
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': -20.49441174809682, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.49)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: -5.67426413431
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', 'right'), 'deadline': 10, 't': 10, 'action': None, 'reward': -5.674264134306023, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.67)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 2.71630598648
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.716305986475693, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.72)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 2.03459081448
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.0345908144780047, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.03)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: left, reward: -9.12883925155
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': -9.128839251549334, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.13)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 0.127586182767
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 0.12758618276675537, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent drove right instead of left. (rewarded 0.13)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: -0.396107865403
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -0.3961078654026755, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded -0.40)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 1.28235518221
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.282355182211839, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.28)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: forward, reward: 0.0710930980985
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'forward'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.07109309809849484, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'forward')
Agent drove forward instead of left. (rewarded 0.07)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 2.16579582657
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'forward'), 'deadline': 2, 't': 18, 'action': None, 'reward': 2.165795826568899, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.17)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: forward, reward: -9.30502674845
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -9.305026748453608, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.31)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 20
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (3, 4), deadline = 30
Simulating trial. . .
epsilon = 0.8187; alpha = 0.1000
Simulating trial. . .
epsilon = 0.8187; alpha = 0.1000
Simulating trial. . .
epsilon = 0.8187; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: left, reward: 1.76382888377
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 1.7638288837711154, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove left instead of right. (rewarded 1.76)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 1.39924629533
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.399246295326284, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove right instead of left. (rewarded 1.40)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 1.69580452055
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 1.6958045205465644, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.70)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: -9.50539367904
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'left', 'reward': -9.505393679042662, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.51)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: 2.05185112951
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.0518511295088686, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.05)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: 0.0861279374587
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 0.08612793745870262, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 0.09)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: -9.37907407358
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 24, 't': 6, 'action': 'left', 'reward': -9.3790740735762, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.38)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: -10.0430728957
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 23, 't': 7, 'action': 'left', 'reward': -10.043072895676726, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.04)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.746969685
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.746969685002651, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.75)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: 1.25700892477
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.2570089247734386, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.26)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: right, reward: 1.60620714855
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.6062071485467604, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove right instead of forward. (rewarded 1.61)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: forward, reward: 0.329991724504
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 0.32999172450384606, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.33)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: forward, reward: 0.657064198098
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', None), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': 0.6570641980978756, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', None)
Agent drove forward instead of left. (rewarded 0.66)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 0.716027162791
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': None, 'reward': 0.7160271627911865, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.72)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: 0.128453372896
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'left'), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 0.12845337289596492, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'left')
Agent drove right instead of left. (rewarded 0.13)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: -40.0606587975
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 15, 't': 15, 'action': 'left', 'reward': -40.06065879752785, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.06)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 0.0324953199569
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 14, 't': 16, 'action': None, 'reward': 0.03249531995691157, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.03)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: right, reward: 1.26531661821
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'forward'), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 1.2653166182067603, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 1.27)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 1.43309362637
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 1.4330936263737195, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.43)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: -9.15918402069
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': -9.159184020691523, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -9.16)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: left, reward: -40.6742242405
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 10, 't': 20, 'action': 'left', 'reward': -40.67422424054542, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.67)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 0.528116174096
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 9, 't': 21, 'action': None, 'reward': 0.5281161740961714, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.53)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 2.23529838617
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': 2.235298386170321, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.24)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 1.59827578731
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 7, 't': 23, 'action': None, 'reward': 1.5982757873066238, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.60)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 1.21791447485
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 1.2179144748494168, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.22)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 2.31784917798
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 25, 'action': 'left', 'reward': 2.3178491779751393, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.32)
13% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 21
\-------------------------
Environment.reset(): Trial set up with start = (5, 2), destination = (3, 6), deadline = 20
Simulating trial. . .
epsilon = 0.8106; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: right, reward: 1.72988783388
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'right'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.7298878338776456, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'right')
Agent followed the waypoint right. (rewarded 1.73)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 1.24901855351
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.2490185535086846, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 1.25)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: left, reward: 1.06552480105
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.06552480104687, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.07)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 1.45457425614
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.4545742561434927, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.45)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: -4.70118239304
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': -4.701182393036969, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.70)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.04819382001
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.0481938200145988, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.05)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: -9.50019991859
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': -9.500199918592108, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent attempted driving left through a red light. (rewarded -9.50)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: -10.3882081397
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': -10.388208139681144, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent attempted driving left through a red light. (rewarded -10.39)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: -10.3028878874
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -10.302887887398606, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -10.30)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 2.51369984038
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 2.5136998403803834, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.51)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: 1.04220876657
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.0422087665662672, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.04)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 22
\-------------------------
Environment.reset(): Trial set up with start = (7, 2), destination = (2, 4), deadline = 25
Simulating trial. . .
epsilon = 0.8025; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: -5.72078339018
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 25, 't': 0, 'action': None, 'reward': -5.720783390184011, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.72)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: 1.36563461736
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 1.3656346173555265, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 1.37)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: 1.48368458309
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 1.483684583091796, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.48)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.18598047271
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.1859804727103542, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.19)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 1.70362536148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.7036253614836756, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.70)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: -9.49837067548
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': -9.4983706754794, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.50)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: -40.4054136993
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'left'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': -40.40541369927166, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.41)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 1.23054001059
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.2305400105899764, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.23)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: -9.14218496733
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -9.142184967334055, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.14)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 2.00965813481
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 2.009658134809795, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.01)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: -4.82631668397
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 15, 't': 10, 'action': None, 'reward': -4.826316683966622, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.83)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: 1.43321106311
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.4332110631101243, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.43)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 1.70972288395
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.709722883946875, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.71)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: left, reward: 1.6906126537
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 1.6906126537031745, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.69)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 2.09857924332
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 2.0985792433176282, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.10)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: -5.03327679799
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': -5.0332767979948425, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.03)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: -5.792753764
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 9, 't': 16, 'action': None, 'reward': -5.792753763997952, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.79)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: forward, reward: -40.0128849674
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': -40.012884967394534, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.01)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: left, reward: -9.25620621843
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 7, 't': 18, 'action': 'left', 'reward': -9.256206218429568, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent attempted driving left through a red light. (rewarded -9.26)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: left, reward: -19.4384532393
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 6, 't': 19, 'action': 'left', 'reward': -19.438453239301325, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.44)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: forward, reward: -0.428228929368
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 5, 't': 20, 'action': 'forward', 'reward': -0.4282289293679824, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded -0.43)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: forward, reward: 0.557050178611
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 0.5570501786109848, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent drove forward instead of right. (rewarded 0.56)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: right, reward: 1.56144337724
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', 'forward'), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 1.5614433772367613, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', 'forward')
Agent followed the waypoint right. (rewarded 1.56)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: left, reward: -0.431176577452
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 2, 't': 23, 'action': 'left', 'reward': -0.43117657745236704, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded -0.43)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: forward, reward: 1.42255433918
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': 1.4225543391793216, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.42)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 23
\-------------------------
Environment.reset(): Trial set up with start = (2, 7), destination = (7, 5), deadline = 25
Simulating trial. . .
epsilon = 0.7945; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7945; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 1.36754797443
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.367547974432761, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.37)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: left, reward: 2.12314845876
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 2.12314845875863, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 2.12)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: 1.15626182985
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 1.156261829847088, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.16)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: -4.53577666216
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': -4.535776662156794, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.54)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: -20.4842601164
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': -20.484260116440833, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.48)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 0.430670435927
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.4306704359266592, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.43)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: 1.64878894189
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.6487889418876729, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 1.65)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: left, reward: 2.66017217656
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 2.660172176563515, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.66)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: -10.7254337178
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -10.725433717776607, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.73)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 2.29045688371
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.2904568837078183, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 2.29)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: left, reward: -9.36625152312
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': -9.366251523118384, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.37)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: left, reward: 1.09506560498
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.0950656049793912, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.10)
52% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 24
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (1, 3), deadline = 20
Simulating trial. . .
epsilon = 0.7866; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7866; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7866; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: -10.1311168222
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -10.131116822154269, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent attempted driving left through a red light. (rewarded -10.13)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: 0.853375186692
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.8533751866924945, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.85)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: -10.2218843129
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': -10.221884312864995, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.22)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: left, reward: -9.07173228796
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -9.071732287962147, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.07)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: right, reward: 1.45099423055
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.4509942305537606, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent drove right instead of left. (rewarded 1.45)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: -10.4710560797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': -10.471056079708388, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.47)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 0.0777432235837
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 0.07774322358365882, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.08)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 1.82767725131
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.827677251312839, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.83)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: -39.3450914398
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -39.34509143979741, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.35)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: -5.55505163768
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': None, 'reward': -5.55505163768354, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.56)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: -0.0789678119828
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': -0.07896781198281311, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded -0.08)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: left, reward: -9.39743323657
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': -9.397433236574418, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.40)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: None, reward: 1.33718236114
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.3371823611373017, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.34)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: None, reward: -4.75871681782
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': -4.7587168178224815, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.76)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: forward, reward: 0.0879388317712
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 0.08793883177117989, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent drove forward instead of right. (rewarded 0.09)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: forward, reward: -10.8113315952
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -10.811331595228998, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.81)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: left, reward: -40.4400885929
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': -40.440088592861976, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.44)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: -4.89269999958
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 3, 't': 17, 'action': None, 'reward': -4.892699999581604, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.89)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: left, reward: -19.8056184275
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'forward'), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -19.805618427506328, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.81)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: left, reward: -39.0535175167
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': -39.053517516697276, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.05)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 25
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (7, 7), deadline = 20
Simulating trial. . .
epsilon = 0.7788; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: forward, reward: 0.47380525226
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 0.47380525225993186, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 0.47)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: -4.71062377575
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 19, 't': 1, 'action': None, 'reward': -4.710623775751573, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.71)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: 1.20131283016
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.2013128301640001, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.20)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: -9.72436431224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -9.724364312237725, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.72)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 2.66974900141
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.669749001407193, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.67)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 2.2221522807
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.222152280697223, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.22)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: forward, reward: 0.810043989477
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 0.8100439894767879, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent drove forward instead of left. (rewarded 0.81)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 0.64221294243
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.64221294242973, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 0.64)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.25999586676
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.2599958667640399, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.26)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: forward, reward: -10.9020516021
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, 'right'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': -10.902051602066845, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'right')
Agent attempted driving forward through a red light. (rewarded -10.90)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.305007832
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.305007832000533, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.31)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 0.619638396733
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 0.6196383967328447, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.62)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: 0.00288856900861
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 0.0028885690086080107, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove left instead of right. (rewarded 0.00)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: -9.34587072779
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': -9.34587072779042, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -9.35)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: -9.83144897177
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': -9.831448971771179, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.83)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: -0.265737325206
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -0.2657373252064714, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded -0.27)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: -10.1825321431
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': -10.18253214308385, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.18)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: forward, reward: 0.308070326269
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.30807032626930597, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent drove forward instead of right. (rewarded 0.31)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 0.28476633653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.28476633652998395, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.28)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 0.434765764165
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.43476576416496115, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.43)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 26
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (3, 2), deadline = 20
Simulating trial. . .
epsilon = 0.7711; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7711; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7711; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.10708332084
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.1070833208358108, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.11)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.1930051499
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.1930051498987002, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.19)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.2790273016
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.2790273016038372, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.28)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: forward, reward: -9.13166945535
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': -9.131669455352936, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.13)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: 0.996132115217
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 0.9961321152166505, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.00)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 0.901111734897
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.901111734896975, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent drove right instead of forward. (rewarded 0.90)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 2.63246863246
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.6324686324628743, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.63)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: forward, reward: -9.43718094765
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -9.437180947647228, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.44)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: forward, reward: -10.1887187447
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -10.188718744727513, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.19)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: right, reward: 0.36238013996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.3623801399595934, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.36)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: left, reward: -9.49509847868
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -9.495098478680216, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.50)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: left, reward: -40.4344449791
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': -40.434444979131165, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.43)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 1.3815636869
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.3815636869045869, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.38)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: right, reward: 1.47201920732
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.4720192073152831, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 1.47)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: forward, reward: -0.375272934605
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': -0.3752729346051157, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent drove forward instead of right. (rewarded -0.38)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: right, reward: 1.64462749736
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.6446274973575545, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.64)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: -39.3067408134
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': -39.3067408133525, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.31)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: -9.51914049198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -9.519140491976884, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.52)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: -10.1121180674
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': -10.112118067417851, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.11)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: 0.6911722629
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 0.6911722628997108, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.69)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 27
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (7, 4), deadline = 20
Simulating trial. . .
epsilon = 0.7634; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7634; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 1.44461630501
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.4446163050140626, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.44)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 1.16261837907
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.1626183790676516, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.16)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 0.725691444112
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.7256914441117192, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 0.73)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 1.95202955124
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.952029551237059, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.95)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: forward, reward: -10.8616297903
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': -10.861629790280446, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.86)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: left, reward: -10.0096593699
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': -10.009659369948924, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.01)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: left, reward: 1.45665684813
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.4566568481312223, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove left instead of right. (rewarded 1.46)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: left, reward: 2.65303582685
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 2.653035826845537, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.65)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 0.0642758341953
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.06427583419530924, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.06)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 1.77901790359
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.7790179035885179, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 1.78)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 2.36756275525
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 2.3675627552474428, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.37)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: -9.31182868203
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': -9.311828682034163, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.31)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: left, reward: -9.45601990014
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'left', 'left'), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -9.456019900141872, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', 'left')
Agent attempted driving left through a red light. (rewarded -9.46)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: forward, reward: 1.10116456703
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.1011645670340653, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'left')
Agent drove forward instead of right. (rewarded 1.10)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: -5.24964894048
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': -5.2496489404784565, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.25)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 0.719571117119
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'forward'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 0.7195711171189894, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 0.72)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 1.3523986152
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.3523986151993521, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.35)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: -0.4690206118
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': -0.46902061179996923, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent drove right instead of left. (rewarded -0.47)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 1.35489619089
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 1.35489619088808, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.35)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 1.11915061608
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 1.119150616082733, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.12)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 28
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (3, 2), deadline = 25
Simulating trial. . .
epsilon = 0.7558; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7558; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7558; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7558; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.49009381684
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.490093816841605, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.49)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.78123636261
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.7812363626053036, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.78)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: -10.1816181677
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': -10.181618167655369, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', None)
Agent attempted driving forward through a red light. (rewarded -10.18)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.13220694348
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.1322069434755564, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.13)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: -10.9512072473
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -10.951207247344316, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.95)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 0.50047290654
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.500472906540303, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.50)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: -9.12985791472
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -9.129857914716263, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.13)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: forward, reward: 0.538533425449
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 0.5385334254492837, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.54)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: -5.0365001322
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'left', 'right'), 'deadline': 17, 't': 8, 'action': None, 'reward': -5.036500132199045, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.04)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: left, reward: -10.3787802092
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': -10.378780209209404, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -10.38)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 1.82695678904
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.826956789041174, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.83)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: forward, reward: -10.2337643364
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': -10.233764336435904, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.23)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: left, reward: -10.9994115345
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': 'left', 'reward': -10.999411534539552, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -11.00)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: left, reward: 2.08358337318
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 2.083583373181784, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.08)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: left, reward: -9.30202851636
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -9.302028516363693, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.30)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: right, reward: 1.38510391094
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.385103910936197, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.39)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: forward, reward: -10.187342753
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': -10.187342752952947, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.19)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: forward, reward: 1.44058870251
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 1.4405887025053907, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.44)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: 1.85950294508
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 7, 't': 18, 'action': 'left', 'reward': 1.8595029450764367, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 1.86)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: -39.209736306
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 6, 't': 19, 'action': 'left', 'reward': -39.209736306032134, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.21)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: -40.0949513644
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 5, 't': 20, 'action': 'left', 'reward': -40.094951364399975, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.09)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: -5.34559845652
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': None, 'reward': -5.345598456518901, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.35)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: 1.03779554428
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 3, 't': 22, 'action': 'left', 'reward': 1.0377955442818787, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.04)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 0.851369212135
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 23, 'action': 'right', 'reward': 0.8513692121349437, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 0.85)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 0.998862602749
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 1, 't': 24, 'action': 'right', 'reward': 0.9988626027492826, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, 'right')
Agent drove right instead of forward. (rewarded 1.00)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 29
\-------------------------
Environment.reset(): Trial set up with start = (4, 3), destination = (6, 6), deadline = 25
Simulating trial. . .
epsilon = 0.7483; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7483; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 2.06069636572
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.060696365715269, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 2.06)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: 2.75776801942
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.757768019418748, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', 'left')
Agent followed the waypoint right. (rewarded 2.76)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 1.08431230109
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.0843123010884523, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.08)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: 0.968733068169
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 0.9687330681694142, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.97)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: -5.44272703826
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 21, 't': 4, 'action': None, 'reward': -5.442727038263893, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.44)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 0.495671494153
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'left'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.4956714941529753, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'left')
Agent drove right instead of left. (rewarded 0.50)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: left, reward: 1.74205886455
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.7420588645493114, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent drove left instead of right. (rewarded 1.74)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 2.59022752576
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.590227525757264, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.59)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: forward, reward: -9.20888902745
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -9.208889027445315, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.21)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: left, reward: -20.6305770191
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'left', 'reward': -20.630577019069246, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.63)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: -4.07125321974
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': -4.071253219738696, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.07)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: forward, reward: 1.80087442335
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.8008744233457925, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', None)
Agent drove forward instead of left. (rewarded 1.80)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: -5.84037798306
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': -5.840377983064947, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.84)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 2.37847236847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 12, 't': 13, 'action': None, 'reward': 2.3784723684731457, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.38)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: left, reward: -10.3957675969
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -10.395767596889534, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent attempted driving left through a red light. (rewarded -10.40)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 1.24217075779
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.2421707577896508, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.24)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: -4.34364765251
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': -4.34364765250653, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.34)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: -0.165975932352
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': -0.16597593235161523, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent drove right instead of left. (rewarded -0.17)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: left, reward: -20.7433763802
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 7, 't': 18, 'action': 'left', 'reward': -20.743376380195706, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.74)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: left, reward: 1.68395395931
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 1.683953959314468, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.68)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 0.623180489903
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.6231804899030786, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.62)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: left, reward: 1.42167470671
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 1.4216747067066362, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.42)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: -0.367075771013
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 3, 't': 22, 'action': 'right', 'reward': -0.3670757710131658, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent drove right instead of forward. (rewarded -0.37)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: -10.0690117713
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 2, 't': 23, 'action': 'left', 'reward': -10.0690117713302, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -10.07)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 0.303357099423
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.3033570994230532, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.30)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 30
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (1, 4), deadline = 20
Simulating trial. . .
epsilon = 0.7408; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7408; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7408; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7408; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: right, reward: 1.31997778244
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.3199777824353127, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.32)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: left, reward: 2.74560880666
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 2.745608806663804, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 2.75)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 1.61919224894
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.6191922489363324, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 1.62)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: 1.02765138943
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'forward'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.0276513894327546, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'forward')
Agent drove forward instead of left. (rewarded 1.03)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: 1.37289915724
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.3728991572445868, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.37)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: 1.71396005943
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.7139600594253608, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', 'left')
Agent drove left instead of forward. (rewarded 1.71)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: -5.83369203322
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': None, 'reward': -5.833692033219327, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.83)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 1.74492934104
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.7449293410368822, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 1.74)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: -39.6683108517
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -39.66831085166616, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.67)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: left, reward: 1.69626784699
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.6962678469855281, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove left instead of right. (rewarded 1.70)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: right, reward: -0.0589796390695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': -0.05897963906945425, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded -0.06)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 0.701977255822
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.7019772558222022, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent drove right instead of left. (rewarded 0.70)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: forward, reward: -9.18803453568
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': -9.188034535682116, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.19)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: forward, reward: 0.920448526721
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.9204485267213828, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 0.92)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 1.7639506847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.7639506846976787, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.76)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 2.12146523766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.1214652376629894, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.12)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 2.21140793231
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 4, 't': 16, 'action': None, 'reward': 2.211407932306587, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.21)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: -5.66813457872
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 3, 't': 17, 'action': None, 'reward': -5.668134578720375, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.67)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: -4.14185832969
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 2, 't': 18, 'action': None, 'reward': -4.141858329688732, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.14)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: 2.09614678883
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 2.0961467888290715, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.10)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 31
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (8, 5), deadline = 20
Simulating trial. . .
epsilon = 0.7334; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7334; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7334; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7334; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: left, reward: -9.66811248116
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -9.668112481163949, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -9.67)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: left, reward: -10.7401836723
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -10.740183672290541, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.74)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: right, reward: 2.922774745
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.922774744996561, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 2.92)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 2.43124953471
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.4312495347060055, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.43)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 1.96775619183
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.9677561918320785, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.97)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: left, reward: -20.4293101016
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', 'forward'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': -20.429310101607, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.43)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: 1.32615630751
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.326156307509885, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.33)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 2.56650992817
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.5665099281667887, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.57)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 1.45025070466
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.4502507046557387, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', None)
Agent drove right instead of forward. (rewarded 1.45)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: left, reward: 1.74480436972
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.7448043697245643, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.74)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 32
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (6, 3), deadline = 20
Simulating trial. . .
epsilon = 0.7261; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: left, reward: -10.3025657423
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -10.302565742316286, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.30)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 1.40768358406
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.4076835840612507, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.41)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: -10.7852787918
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': -10.785278791797065, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.79)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: -9.74796043167
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': -9.747960431669812, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.75)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: -9.19552538205
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': -9.195525382054637, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -9.20)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: left, reward: 2.68532144518
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.685321445177804, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.69)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 1.73928441834
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.7392844183432725, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.74)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: -10.0429699644
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -10.042969964396693, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.04)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 2.10120823402
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.101208234019393, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.10)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: -5.22210322539
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': -5.222103225388094, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.22)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: 2.1727408792
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 2.172740879195878, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.17)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 1.42052915227
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.4205291522691899, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.42)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 0.618605450439
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'forward'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.6186054504389236, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'forward')
Agent drove right instead of left. (rewarded 0.62)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: -10.1702449189
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': -10.17024491890813, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.17)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 2.10202808579
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.1020280857898244, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.10)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 0.911943899398
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'right', None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 0.9119438993975808, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'right', None)
Agent followed the waypoint right. (rewarded 0.91)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: forward, reward: -10.9155846864
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': -10.915584686408833, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -10.92)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: -4.35791282631
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 3, 't': 17, 'action': None, 'reward': -4.357912826305942, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.36)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: forward, reward: 0.26263871974
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 0.2626387197397775, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent drove forward instead of right. (rewarded 0.26)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 1.45804735359
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.4580473535875018, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.46)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 33
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (4, 6), deadline = 20
Simulating trial. . .
epsilon = 0.7189; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 2.34411030803
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.344110308028305, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.34)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 1.03500576241
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.0350057624105409, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent drove right instead of forward. (rewarded 1.04)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.53381185201
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'right'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.5338118520100945, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'right')
Agent properly idled at a red light. (rewarded 1.53)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: forward, reward: -10.4936119577
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': -10.493611957731153, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.49)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 2.7631002644
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.7631002643958285, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.76)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: left, reward: 1.53294885129
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.5329488512917449, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.53)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: forward, reward: -9.91713774934
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -9.917137749338625, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent attempted driving forward through a red light. (rewarded -9.92)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 1.03995097023
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.0399509702255725, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.04)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: left, reward: 2.15179734617
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 2.151797346174787, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.15)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 34
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (4, 3), deadline = 25
Simulating trial. . .
epsilon = 0.7118; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 2.14122465051
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.1412246505051353, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.14)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.40846793633
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.4084679363274197, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 1.41)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 2.25807970267
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 2.258079702670691, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.26)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 1.76256843852
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.7625684385234495, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.76)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.44729987383
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.4472998738322524, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.45)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: -10.9340578977
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': -10.934057897651455, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', None)
Agent attempted driving forward through a red light. (rewarded -10.93)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: -10.7823640283
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -10.78236402825805, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.78)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: 0.955182378633
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 0.955182378633066, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.96)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: 2.55839885161
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.558398851612516, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.56)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: -10.1163457461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'left', 'reward': -10.116345746060503, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.12)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: -19.4890764561
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': -19.48907645609936, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.49)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: left, reward: 1.02760029938
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.0276002993834763, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 1.03)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 2.12070542546
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'right'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 2.1207054254582607, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 2.12)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: right, reward: 1.90318146724
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.9031814672436906, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.90)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: left, reward: -0.0717136234771
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -0.071713623477064, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded -0.07)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 2.06741335291
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.0674133529054037, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.07)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: right, reward: 1.60033279907
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 1.6003327990723044, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.60)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 0.630195374869
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 0.6301953748693616, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.63)
28% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 35
\-------------------------
Environment.reset(): Trial set up with start = (5, 5), destination = (1, 7), deadline = 30
Simulating trial. . .
epsilon = 0.7047; alpha = 0.1000
Simulating trial. . .
epsilon = 0.7047; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 1.91828915336
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'right', 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.918289153361247, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'right', 'left')
Agent drove right instead of left. (rewarded 1.92)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: -5.2620230575
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 29, 't': 1, 'action': None, 'reward': -5.262023057495978, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.26)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: left, reward: -20.2041792509
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'forward'), 'deadline': 28, 't': 2, 'action': 'left', 'reward': -20.20417925092128, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.20)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: left, reward: -20.6785080348
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'right'), 'deadline': 27, 't': 3, 'action': 'left', 'reward': -20.678508034832404, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.68)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 2.34326763356
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.343267633559205, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.34)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: 2.26808832148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 2.2680883214794028, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.27)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: right, reward: 1.69566347963
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 1.6956634796333674, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.70)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: right, reward: -20.0412913587
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'right'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': -20.0412913586756, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'right')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.04)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: forward, reward: -9.61126532867
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': -9.611265328674811, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.61)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: -4.4231931667
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 21, 't': 9, 'action': None, 'reward': -4.423193166703869, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.42)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: left, reward: 2.73462669727
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 2.734626697269614, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 2.73)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 2.74328274363
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.743282743630893, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.74)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: forward, reward: -39.6233850285
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': -39.62338502848824, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.62)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 0.927379763563
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 0.927379763563458, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.93)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 2.20772369562
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 14, 'action': None, 'reward': 2.2077236956190798, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.21)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: forward, reward: -10.7201773379
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': -10.720177337924204, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -10.72)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 1.24750796889
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 14, 't': 16, 'action': None, 'reward': 1.2475079688854895, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.25)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: forward, reward: -0.212298402267
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': -0.21229840226717234, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded -0.21)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: 0.0730550739815
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 0.07305507398153566, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 0.07)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: -5.83605977284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 11, 't': 19, 'action': None, 'reward': -5.836059772839472, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.84)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 1.95155922446
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 1.9515592244620426, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 1.95)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: None, reward: -5.18057556759
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 9, 't': 21, 'action': None, 'reward': -5.180575567585412, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.18)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 1.95112056483
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 1.9511205648257688, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.95)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: 2.30615781805
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 7, 't': 23, 'action': 'forward', 'reward': 2.3061578180548072, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.31)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 1.53001191366
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'forward'), 'deadline': 6, 't': 24, 'action': 'forward', 'reward': 1.5300119136636294, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.53)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.91294543761
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 1.912945437613684, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.91)
13% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 36
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (6, 6), deadline = 20
Simulating trial. . .
epsilon = 0.6977; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: forward, reward: 1.71779947193
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.71779947193341, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent drove forward instead of right. (rewarded 1.72)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: right, reward: 1.46897672506
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'left', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.468976725058506, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'left', None)
Agent followed the waypoint right. (rewarded 1.47)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: -40.5251415944
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': -40.525141594391044, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.53)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: left, reward: -9.66489466248
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -9.664894662483134, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.66)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: left, reward: -10.4104751726
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -10.410475172643793, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.41)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: right, reward: 0.125442174286
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.12544217428561333, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.13)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: forward, reward: 0.948355519371
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 0.9483555193708135, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded 0.95)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: right, reward: -0.118980377616
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': -0.1189803776156726, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded -0.12)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: left, reward: 0.719779081594
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 0.7197790815944524, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 0.72)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: 1.15117182323
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.1511718232304855, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded 1.15)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 1.08048012354
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.0804801235351265, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.08)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: 0.431605258688
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.4316052586879857, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.43)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: -40.3131939025
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': -40.31319390247368, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.31)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: left, reward: -19.758423451
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': -19.758423451018956, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.76)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: left, reward: 2.04799152795
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 2.047991527952865, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 2.05)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: 0.823761051115
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 0.8237610511147633, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 0.82)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: left, reward: 0.797370159543
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 0.7973701595428122, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.80)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: None, reward: 1.82463250815
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.8246325081481474, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.82)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: left, reward: -10.5084388435
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -10.508438843483257, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.51)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: left, reward: -20.3723486015
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 1, 't': 19, 'action': 'left', 'reward': -20.37234860152661, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.37)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 37
\-------------------------
Environment.reset(): Trial set up with start = (7, 3), destination = (1, 7), deadline = 20
Simulating trial. . .
epsilon = 0.6907; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: forward, reward: -39.5478245783
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -39.5478245782908, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.55)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: left, reward: -10.7411538984
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -10.74115389838866, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.74)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: left, reward: -9.24687127301
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': -9.246871273011482, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.25)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 2.76737437237
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.7673743723733244, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.77)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: forward, reward: -9.45884102506
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': -9.458841025062442, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.46)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: right, reward: 1.9807324046
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.980732404597261, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.98)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 1.95919087797
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.9591908779722818, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.96)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.54508999073
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.54508999073268, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.55)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.45589084567
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.455890845672877, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.46)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: -9.68121931175
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': -9.68121931175367, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.68)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 0.368435484254
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 0.3684354842537313, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 0.37)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: -5.95550598466
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': -5.955505984664532, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.96)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 0.970162907717
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 0.9701629077172149, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 0.97)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: left, reward: -39.1127495788
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': -39.1127495787546, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.11)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: left, reward: -9.28919400305
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -9.289194003051323, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -9.29)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: -10.3296020463
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -10.32960204628916, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.33)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: -5.07871585937
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 4, 't': 16, 'action': None, 'reward': -5.078715859374368, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.08)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: left, reward: 0.648869474272
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 0.6488694742719534, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 0.65)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: forward, reward: 0.712286987432
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 0.7122869874315993, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', None)
Agent drove forward instead of left. (rewarded 0.71)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 0.939469103577
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.9394691035773847, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.94)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 38
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (1, 5), deadline = 20
Simulating trial. . .
epsilon = 0.6839; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.53009499566
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.5300949956555436, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.53)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: -9.04136782845
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -9.041367828445821, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -9.04)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 0.287996832779
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.2879968327788939, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.29)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: -4.15327293743
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': -4.153272937431929, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.15)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: forward, reward: 1.3838355669
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.3838355669035844, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 1.38)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: left, reward: 2.47922125458
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.4792212545828685, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.48)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: right, reward: 1.17134993761
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.1713499376074803, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent drove right instead of forward. (rewarded 1.17)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.4195916948
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', 'right'), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.4195916947967746, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', 'right')
Agent properly idled at a red light. (rewarded 1.42)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: right, reward: 1.44003218773
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.4400321877263589, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 1.44)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 1.54633879766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.5463387976642275, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.55)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: -9.96187447104
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': -9.961874471037435, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.96)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: left, reward: 0.778687487798
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 0.7786874877983476, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.78)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: left, reward: -9.02858366835
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -9.02858366835213, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -9.03)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.6960905277
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.6960905277009424, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.70)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.72061041964
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.72061041963678, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.72)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.07107515308
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.0710751530824807, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.07)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: 2.22655041856
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 2.2265504185592633, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.23)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: -10.7975527784
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': -10.797552778419574, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.80)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: -39.177114136
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -39.17711413598381, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.18)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.44222733056
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.4422273305623043, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.44)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 39
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (6, 6), deadline = 25
Simulating trial. . .
epsilon = 0.6771; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 1.43069692606
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.4306969260609839, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.43)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: forward, reward: 1.70304315491
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'left'), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 1.703043154908927, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'left')
Agent drove forward instead of left. (rewarded 1.70)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 1.349956365
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.3499563650020079, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 1.35)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: 1.1388417685
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.1388417684959393, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 1.14)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: -40.7152365356
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -40.71523653559041, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.72)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.12682927187
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.1268292718652813, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.13)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: left, reward: 0.267398155854
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 0.267398155854469, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.27)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 1.82620294126
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'right'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.8262029412632033, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 1.83)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 0.110768147065
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.1107681470649583, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.11)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: left, reward: 2.79160973199
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 2.7916097319916533, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.79)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: right, reward: -0.0480732450884
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': -0.04807324508839328, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded -0.05)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: 1.51474832717
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', 'forward'), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.5147483271709292, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.51)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: forward, reward: -9.95775259933
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': -9.957752599334736, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.96)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: right, reward: -0.231848180755
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': -0.23184818075547975, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded -0.23)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: forward, reward: 0.28474465223
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 0.2847446522297913, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.28)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: -4.51169954844
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 10, 't': 15, 'action': None, 'reward': -4.511699548435463, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.51)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: -5.38189740382
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 9, 't': 16, 'action': None, 'reward': -5.3818974038188365, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.38)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 1.63282701921
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 8, 't': 17, 'action': None, 'reward': 1.6328270192137344, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.63)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: forward, reward: -10.9905973222
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': -10.990597322156711, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.99)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: left, reward: -0.190197502179
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 6, 't': 19, 'action': 'left', 'reward': -0.1901975021789214, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent drove left instead of right. (rewarded -0.19)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 0.752770320063
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 0.7527703200634257, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.75)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: -10.1130252291
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': -10.113025229069255, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.11)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.51642827442
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 3, 't': 22, 'action': None, 'reward': 1.5164282744177124, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.52)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: -9.69609129477
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 2, 't': 23, 'action': 'left', 'reward': -9.696091294772403, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -9.70)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: -10.1013128207
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 1, 't': 24, 'action': 'left', 'reward': -10.101312820657789, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.10)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 40
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (6, 7), deadline = 20
Simulating trial. . .
epsilon = 0.6703; alpha = 0.1000
Simulating trial. . .
epsilon = 0.6703; alpha = 0.1000
Simulating trial. . .
epsilon = 0.6703; alpha = 0.1000
Simulating trial. . .
epsilon = 0.6703; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 0.452214917979
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 0.4522149179791106, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.45)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: 1.68663634719
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.6866363471888839, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.69)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 2.38940435078
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.3894043507772644, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.39)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: forward, reward: -40.3449169366
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': -40.3449169366207, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.34)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: left, reward: 1.45864372226
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.4586437222648665, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.46)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: right, reward: 1.79726484464
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.7972648446404385, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 1.80)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: right, reward: 0.0791102869266
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.07911028692658473, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 0.08)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 1.5151315555
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'right'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.5151315555000717, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'right')
Agent followed the waypoint right. (rewarded 1.52)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: left, reward: -10.1670401621
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -10.16704016210502, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -10.17)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: -10.4591026394
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': -10.459102639369554, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.46)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: 2.085728346
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'forward'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 2.085728345998807, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'forward')
Agent followed the waypoint right. (rewarded 2.09)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: forward, reward: 1.92182178409
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.9218217840867213, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 1.92)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: forward, reward: 1.2144441762
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 1.214444176196709, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.21)
35% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 41
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (8, 2), deadline = 20
Simulating trial. . .
epsilon = 0.6637; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: forward, reward: -10.814482221
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -10.8144822210343, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -10.81)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: None, reward: 1.34940179291
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.3494017929106084, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.35)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 2.49539513359
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.495395133585827, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.50)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: -5.54390702613
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 17, 't': 3, 'action': None, 'reward': -5.543907026133466, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.54)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: 1.22629170927
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.2262917092718786, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.23)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: -20.0834176022
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, 'left', 'right'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': -20.083417602240512, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.08)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: -5.54821332494
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 14, 't': 6, 'action': None, 'reward': -5.54821332493562, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.55)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 1.02280458635
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.022804586349326, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.02)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: 2.75650936291
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.756509362914165, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.76)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 0.943472743154
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.9434727431536163, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.94)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.29178798385
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.2917879838478614, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.29)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: 1.49680794317
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 1.496807943168659, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.50)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: -4.61003277598
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 8, 't': 12, 'action': None, 'reward': -4.610032775978171, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.61)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 1.69568323948
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.6956832394838086, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.70)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: forward, reward: 0.0263924477208
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 0.02639244772079674, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.03)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: forward, reward: 0.250366582012
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 0.25036658201208284, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove forward instead of right. (rewarded 0.25)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: left, reward: -39.7848092522
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': -39.784809252203566, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.78)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 2.17867603854
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 2.178676038541349, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', None)
Agent followed the waypoint right. (rewarded 2.18)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 0.583485247332
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 0.583485247332411, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove forward instead of right. (rewarded 0.58)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 1.91368015978
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 1.9136801597780513, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.91)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 42
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (8, 3), deadline = 25
Simulating trial. . .
epsilon = 0.6570; alpha = 0.1000
Simulating trial. . .
epsilon = 0.6570; alpha = 0.1000
Simulating trial. . .
epsilon = 0.6570; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: 1.00755585897
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.0075558589656652, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.01)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: 0.216322702221
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.21632270222143757, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.22)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: forward, reward: -9.05000205836
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': -9.050002058362244, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.05)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 1.010182908
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.0101829080037346, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.01)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 1.45399272781
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.4539927278077782, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.45)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: 2.49124939622
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 2.491249396216393, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.49)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: forward, reward: -9.33925719454
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -9.339257194537575, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -9.34)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 1.16031126878
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.1603112687829034, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.16)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: left, reward: 1.58702706383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 1.5870270638291637, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.59)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: 2.08817069055
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 2.0881706905473822, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.09)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: 1.55031157043
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.5503115704337795, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove right instead of forward. (rewarded 1.55)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: left, reward: 1.21281017306
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.2128101730596375, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.21)
52% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 43
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (7, 4), deadline = 20
Simulating trial. . .
epsilon = 0.6505; alpha = 0.1000
Simulating trial. . .
epsilon = 0.6505; alpha = 0.1000
Simulating trial. . .
epsilon = 0.6505; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: -10.8264460772
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -10.826446077218566, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.83)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.65618545667
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.656185456669353, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.66)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 2.05026131119
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.0502613111942587, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.05)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 2.40247112734
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 2.402471127337668, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.40)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 2.08298537811
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.0829853781099508, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.08)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 1.85803540937
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.8580354093691676, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.86)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: left, reward: 1.44356409086
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.4435640908621143, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.44)
65% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 44
\-------------------------
Environment.reset(): Trial set up with start = (7, 6), destination = (8, 3), deadline = 20
Simulating trial. . .
epsilon = 0.6440; alpha = 0.1000
Simulating trial. . .
epsilon = 0.6440; alpha = 0.1000
Simulating trial. . .
epsilon = 0.6440; alpha = 0.1000
Simulating trial. . .
epsilon = 0.6440; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 1.31382253349
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'right'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.31382253349208, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'right')
Agent drove right instead of left. (rewarded 1.31)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: -9.84339606211
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -9.843396062107457, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.84)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: -9.95367094173
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': -9.95367094172869, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.95)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 0.699059831912
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 0.6990598319117308, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove right instead of left. (rewarded 0.70)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 2.70404370399
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.704043703994672, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.70)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 2.70584745094
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.705847450943587, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.71)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: -5.78632142474
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'forward'), 'deadline': 14, 't': 6, 'action': None, 'reward': -5.786321424737015, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.79)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 0.105385339528
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.10538533952820583, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.11)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: -19.3561196149
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': -19.356119614923337, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.36)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: 1.47436993089
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.4743699308888176, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.47)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: -40.2933394857
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'left', 'right'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -40.29333948573894, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.29)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: 2.44266038608
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 2.4426603860802745, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.44)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: 1.52058323078
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 1.5205832307755551, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent drove left instead of forward. (rewarded 1.52)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: left, reward: 1.24151080382
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.2415108038157814, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 1.24)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.15805183081
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.1580518308111263, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.16)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 0.876072908745
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.876072908745358, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.88)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 0.130798465843
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.13079846584287047, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.13)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: forward, reward: 0.459837175798
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.4598371757979086, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 0.46)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: left, reward: -10.958019558
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -10.95801955801782, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -10.96)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: 0.554821000646
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 0.5548210006455495, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.55)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 45
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (6, 7), deadline = 20
Simulating trial. . .
epsilon = 0.6376; alpha = 0.1000
Simulating trial. . .
epsilon = 0.6376; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: forward, reward: -9.32633599256
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -9.326335992562589, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.33)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 2.76630697252
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.766306972520133, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.77)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 2.81065228854
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.8106522885431575, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.81)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 2.49834393086
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.498343930859117, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.50)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: forward, reward: 0.551058790259
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 0.5510587902588903, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded 0.55)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: 1.86251723553
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.8625172355302726, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.86)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: -19.3377964659
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': -19.337796465918675, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.34)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: forward, reward: 1.82159522499
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.8215952249855378, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.82)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 0.948933697514
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 0.9489336975136167, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 0.95)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 1.22860064668
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.2286006466845631, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.23)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: 0.0633450549412
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'left'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 0.06334505494115306, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'left')
Agent drove forward instead of left. (rewarded 0.06)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 1.1978813557
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.1978813557023553, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.20)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: -10.8436130367
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': -10.843613036680011, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.84)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: forward, reward: 1.36477268132
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.3647726813167051, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.36)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: left, reward: -19.8166602457
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, 'right', 'right'), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -19.816660245652457, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.82)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 1.21110385751
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.2111038575052548, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.21)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: forward, reward: -10.7437660355
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': -10.743766035545216, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.74)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: -4.61738130237
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': -4.6173813023671375, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.62)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: 2.10047283753
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': 2.1004728375321977, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.10)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: -9.8157919995
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', 'forward'), 'deadline': 1, 't': 19, 'action': 'left', 'reward': -9.815791999500837, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', 'right', 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -9.82)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 46
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (4, 2), deadline = 20
Simulating trial. . .
epsilon = 0.6313; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 0.0349563707153
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 0.03495637071527202, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.03)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 1.00811205471
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', 'right'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.008112054714868, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', 'right')
Agent followed the waypoint forward. (rewarded 1.01)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 0.495835345983
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'right'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.49583534598347645, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'right')
Agent drove right instead of forward. (rewarded 0.50)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: -4.80207645122
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 17, 't': 3, 'action': None, 'reward': -4.802076451221479, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.80)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: 0.870707146428
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 0.8707071464275683, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 0.87)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: forward, reward: 0.499080656034
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 0.49908065603448337, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'forward')
Agent drove forward instead of left. (rewarded 0.50)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 1.75655588031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'forward'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.7565558803072823, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.76)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 2.12537549933
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.125375499330178, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.13)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: forward, reward: -40.8893226035
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -40.88932260354246, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.89)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 0.904165876675
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.9041658766745151, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent drove right instead of left. (rewarded 0.90)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 0.979146613884
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.9791466138835505, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.98)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: -5.66281217936
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 9, 't': 11, 'action': None, 'reward': -5.662812179359012, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.66)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: 1.34491709837
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 1.344917098365741, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.34)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 0.803595488065
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 0.8035954880649678, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 0.80)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: -10.6481725817
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -10.64817258171855, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.65)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 1.86493065915
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.8649306591469619, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.86)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.05591095197
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.0559109519693264, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.06)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: -4.5443870069
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 3, 't': 17, 'action': None, 'reward': -4.5443870068984245, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.54)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: -5.45673434769
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': -5.456734347694202, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.46)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 0.516391293745
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.5163912937448225, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.52)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 47
\-------------------------
Environment.reset(): Trial set up with start = (4, 7), destination = (6, 4), deadline = 25
Simulating trial. . .
epsilon = 0.6250; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: -9.75566306062
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 25, 't': 0, 'action': 'left', 'reward': -9.75566306061913, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent attempted driving left through a red light. (rewarded -9.76)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: -10.9716063549
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -10.971606354897695, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.97)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.34071122315
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.3407112231534892, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.34)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: -10.6017757401
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': 'left', 'reward': -10.601775740056539, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.60)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.34346735285
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.343467352846819, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.34)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: 1.5959416225
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.5959416225024476, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded 1.60)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: left, reward: -40.9661764306
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': -40.96617643055619, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.97)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: left, reward: -9.69239570036
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': -9.692395700362717, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -9.69)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: left, reward: -40.691210514
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': -40.6912105140216, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.69)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.30668477966
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.3066847796580268, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.31)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.12950359004
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.129503590040323, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.13)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 1.00084089327
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.0008408932680024, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove right instead of left. (rewarded 1.00)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 1.45846922008
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.4584692200771454, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.46)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: forward, reward: -10.9254820887
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': -10.925482088657322, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.93)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 2.31120047367
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.3112004736695955, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.31)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 1.21842665648
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.218426656482169, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.22)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 2.48931395051
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 2.489313950509844, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.49)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: left, reward: 1.45906197507
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 1.459061975065781, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.46)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: forward, reward: -40.4696729628
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': -40.46967296283443, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.47)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: 2.46680950458
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 2.4668095045750285, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 2.47)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: 0.583035908616
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 5, 't': 20, 'action': 'forward', 'reward': 0.5830359086164552, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded 0.58)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.59147136898
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 4, 't': 21, 'action': None, 'reward': 1.591471368984808, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.59)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.85296220334
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 3, 't': 22, 'action': None, 'reward': 1.8529622033378694, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.85)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.04862226317
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.04862226317007, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.05)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 1.62711530731
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 1, 't': 24, 'action': 'right', 'reward': 1.6271153073122717, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.63)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 48
\-------------------------
Environment.reset(): Trial set up with start = (2, 6), destination = (5, 5), deadline = 20
Simulating trial. . .
epsilon = 0.6188; alpha = 0.1000
Simulating trial. . .
epsilon = 0.6188; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 1.9329178185
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.9329178184973308, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.93)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: right, reward: 1.01256166826
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.0125616682630076, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.01)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: left, reward: -40.8733142156
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': -40.87331421556761, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.87)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: forward, reward: -10.7980658999
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': -10.798065899918925, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.80)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 2.78406463151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.7840646315119626, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.78)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.89951262186
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.8995126218564309, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.90)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: -4.04250500478
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': -4.042505004782921, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.04)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: left, reward: 0.779352585329
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 0.7793525853291949, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.78)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: 1.20731812332
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.2073181233196784, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent drove left instead of right. (rewarded 1.21)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: -10.2666109294
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': -10.266610929420025, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.27)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: -40.4214978272
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'forward'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -40.421497827177454, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.42)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: -0.166282950662
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', 'forward'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': -0.1662829506624377, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', 'forward')
Agent drove right instead of left. (rewarded -0.17)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: left, reward: -9.25942195043
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -9.259421950425661, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent attempted driving left through a red light. (rewarded -9.26)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: -5.89878807815
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': -5.8987880781490585, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.90)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: forward, reward: 0.912831133374
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 0.9128311333744975, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.91)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: -9.65640095951
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', 'right', None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -9.656400959511458, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', None)
Agent attempted driving left through a red light. (rewarded -9.66)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: 1.20325813394
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'right'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.2032581339446702, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'right')
Agent followed the waypoint right. (rewarded 1.20)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: -0.445824816873
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': -0.44582481687349407, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent drove right instead of forward. (rewarded -0.45)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: left, reward: 0.661812839723
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': 0.66181283972315, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 0.66)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 0.0934385907711
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.09343859077105454, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.09)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 49
\-------------------------
Environment.reset(): Trial set up with start = (2, 6), destination = (6, 2), deadline = 30
Simulating trial. . .
epsilon = 0.6126; alpha = 0.1000
Simulating trial. . .
epsilon = 0.6126; alpha = 0.1000
Simulating trial. . .
epsilon = 0.6126; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: 0.0836200096695
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 0.08362000966946859, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 0.08)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 0.276890760753
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 0.2768907607530612, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.28)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: 0.782077947695
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 0.7820779476953469, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 0.78)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: 0.0196597965247
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 0.01965979652471128, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 0.02)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 1.30031098359
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.3003109835922972, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.30)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: forward, reward: 1.91615511898
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.916155118978455, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.92)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: left, reward: 0.152480638502
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 0.15248063850230287, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.15)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: forward, reward: -10.8372771667
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': -10.837277166662476, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.84)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: forward, reward: -9.48431371828
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': -9.484313718276889, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.48)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 2.00830802018
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 2.0083080201808805, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.01)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: -39.1399728756
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': -39.139972875605196, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.14)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: left, reward: 1.68157002254
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 1.6815700225386285, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.68)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 2.06380566416
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.0638056641555007, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.06)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: right, reward: 2.04444249915
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 2.044442499153421, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.04)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 2.78169890374
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 16, 't': 14, 'action': None, 'reward': 2.7816989037386217, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.78)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 1.83069995216
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.8306999521574174, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.83)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: -0.120762037251
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 16, 'action': 'right', 'reward': -0.1207620372510021, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded -0.12)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 1.70807544003
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.7080754400266118, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.71)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 1.07546392076
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.0754639207589836, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.08)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: right, reward: 1.06006032114
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 1.0600603211401034, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.06)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 1.33914051887
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 20, 'action': 'forward', 'reward': 1.3391405188716259, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.34)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: 0.768991995953
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 9, 't': 21, 'action': 'left', 'reward': 0.7689919959529139, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 0.77)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 0.594778247616
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 8, 't': 22, 'action': None, 'reward': 0.5947782476162045, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.59)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: forward, reward: 0.322550605087
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'forward'), 'deadline': 7, 't': 23, 'action': 'forward', 'reward': 0.3225506050871626, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'forward')
Agent drove forward instead of left. (rewarded 0.32)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: 1.48727889301
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 6, 't': 24, 'action': 'left', 'reward': 1.4872788930142447, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 1.49)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 1.08808010742
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 1.088080107419142, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 1.09)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: 1.49894711181
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 4, 't': 26, 'action': 'left', 'reward': 1.4989471118120552, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.50)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 0.605413366636
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 27, 'action': None, 'reward': 0.6054133666364865, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.61)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.79014637829
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 2, 't': 28, 'action': None, 'reward': 1.7901463782904805, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.79)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: -0.614369651521
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 1, 't': 29, 'action': 'left', 'reward': -0.6143696515212844, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded -0.61)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 50
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (8, 4), deadline = 30
Simulating trial. . .
epsilon = 0.6065; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 1.37372404601
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.3737240460116886, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.37)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: left, reward: -39.0502172191
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 29, 't': 1, 'action': 'left', 'reward': -39.05021721912644, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.05)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 2.68436279429
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.684362794287958, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.68)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: 2.00552555251
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'left', 'reward': 2.0055255525139257, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.01)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: -10.0437013022
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': -10.043701302167033, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.04)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.88375935418
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.8837593541834226, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.88)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: -20.9515727889
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 24, 't': 6, 'action': 'left', 'reward': -20.95157278886556, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.95)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: -0.030519905662
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 23, 't': 7, 'action': 'right', 'reward': -0.030519905661969937, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove right instead of forward. (rewarded -0.03)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: -10.1813661779
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': -10.181366177922083, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.18)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 1.076188998
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'right'), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.0761889980018025, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'right')
Agent properly idled at a red light. (rewarded 1.08)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: -9.78257989052
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': -9.78257989051987, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.78)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: 2.35806340822
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 2.3580634082227445, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.36)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.90526700576
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.9052670057598482, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.91)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: left, reward: 1.022234106
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 1.0222341060011306, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 1.02)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: forward, reward: 1.69916014845
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'right'), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': 1.6991601484468497, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'right')
Agent drove forward instead of right. (rewarded 1.70)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: right, reward: 1.3628913501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.3628913500983255, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.36)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 2.58787118807
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 16, 'action': None, 'reward': 2.587871188073998, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.59)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: right, reward: -0.173344356135
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 13, 't': 17, 'action': 'right', 'reward': -0.17334435613530474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent drove right instead of forward. (rewarded -0.17)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.23492201896
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.234922018955982, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.23)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: forward, reward: 1.3508405591
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': 1.3508405590991281, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.35)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: left, reward: 0.758355667077
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 20, 'action': 'left', 'reward': 0.7583556670768112, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.76)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 0.944034935446
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 0.9440349354456912, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.94)
27% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 51
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (4, 2), deadline = 20
Simulating trial. . .
epsilon = 0.6005; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 2.2344090604
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.234409060403516, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.23)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 1.46054581799
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.4605458179919601, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 1.46)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 2.46806939339
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.4680693933897704, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 2.47)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 1.57212521476
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.5721252147602809, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.57)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: forward, reward: 0.0215603376563
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 0.021560337656337913, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.02)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 0.98173543034
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 0.9817354303396093, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.98)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: left, reward: 1.29897341929
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.298973419290775, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.30)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 1.79677106883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.7967710688348892, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.80)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: forward, reward: 0.147950462556
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 0.1479504625556224, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.15)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: forward, reward: 0.45797706258
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.45797706257975723, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded 0.46)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: left, reward: -10.3059475198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -10.30594751976713, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent attempted driving left through a red light. (rewarded -10.31)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 2.20785070938
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.20785070937803, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.21)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.29789347367
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.2978934736688825, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.30)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: left, reward: 1.32477348543
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.324773485425591, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.32)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: 1.45017929962
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'right'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.4501792996192222, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'right')
Agent drove right instead of forward. (rewarded 1.45)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: -0.123906993729
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': -0.12390699372925384, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded -0.12)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 1.36874720538
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.3687472053753538, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.37)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.08072839249
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.0807283924909974, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.08)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 1.7134297266
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 1.713429726598275, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.71)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: -10.2831609237
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -10.283160923746907, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.28)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 52
\-------------------------
Environment.reset(): Trial set up with start = (2, 2), destination = (5, 4), deadline = 25
Simulating trial. . .
epsilon = 0.5945; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: 1.56900615505
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'left'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.5690061550461643, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.57)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 0.589080651436
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.5890806514356057, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent drove right instead of forward. (rewarded 0.59)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 1.93523249989
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.9352324998860326, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.94)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 0.718208953206
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 0.7182089532063476, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 0.72)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: -39.6415667183
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -39.64156671825784, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.64)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 1.54397800756
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.5439780075636902, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.54)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 0.721480336798
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 0.721480336798484, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.72)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 2.74167084052
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.7416708405206958, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.74)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 0.730285852233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.7302858522330963, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.73)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.09955453316
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.0995545331550949, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.10)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: -10.1434806654
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': -10.143480665358627, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.14)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 0.628392230869
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 0.6283922308694704, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.63)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 0.375428497914
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 0.37542849791447064, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent drove right instead of left. (rewarded 0.38)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: -0.0293015558288
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': -0.02930155582877103, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove right instead of forward. (rewarded -0.03)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: -0.021829571172
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'forward'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': -0.0218295711720361, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'forward')
Agent drove right instead of left. (rewarded -0.02)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 1.07317729908
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 1.073177299077042, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.07)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: -4.12198062663
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 9, 't': 16, 'action': None, 'reward': -4.121980626632704, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.12)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: -9.73503428315
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': -9.735034283148204, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.74)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 1.23078983726
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 1.2307898372590134, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.23)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: 0.972585104948
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 0.9725851049483949, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 0.97)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 0.638376467242
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.6383764672423697, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 0.64)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: left, reward: 0.785475028585
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 0.7854750285852449, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.79)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 0.400828251126
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.4008282511262886, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 0.40)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 1.03945706631
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.0394570663105496, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.04)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: 0.172655608683
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': 0.1726556086833102, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.17)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 53
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (8, 6), deadline = 20
Simulating trial. . .
epsilon = 0.5886; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: forward, reward: -9.58815180898
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -9.588151808982408, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.59)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: left, reward: -9.31520368342
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -9.315203683421696, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.32)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: left, reward: -9.40965292757
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': 'left', 'reward': -9.409652927566565, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.41)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 1.58368894891
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.583688948907391, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.58)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: left, reward: -10.6610758745
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -10.661075874474431, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.66)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: 1.95788853217
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.957888532170986, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.96)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.37097861168
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.3709786116775193, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.37)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 0.382797704671
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.38279770467091034, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.38)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: -0.000397784788669
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': -0.000397784788668587, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded -0.00)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 1.04623294805
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.0462329480518793, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.05)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: left, reward: -9.52395531605
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -9.523955316045392, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.52)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: 1.41327256335
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.4132725633520202, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.41)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 2.37001681916
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'left'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 2.3700168191641655, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'left')
Agent followed the waypoint forward. (rewarded 2.37)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: -9.2638343907
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': -9.263834390704858, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.26)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 0.561610390571
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.5616103905706502, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 0.56)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: -9.35950058798
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -9.359500587975475, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent attempted driving forward through a red light. (rewarded -9.36)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 0.838881601093
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 0.8388816010931675, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove forward instead of left. (rewarded 0.84)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: left, reward: 1.20653763892
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 1.2065376389246942, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.21)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: left, reward: -20.9580940438
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'right'), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -20.95809404375864, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.96)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 0.57371269577
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.5737126957700328, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 0.57)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 54
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (7, 3), deadline = 30
Simulating trial. . .
epsilon = 0.5827; alpha = 0.1000
Simulating trial. . .
epsilon = 0.5827; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 1.3513129692
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.3513129692041068, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 1.35)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 0.483153199323
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'right'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 0.48315319932332856, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'right')
Agent drove right instead of left. (rewarded 0.48)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: forward, reward: 1.29042500965
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 1.2904250096526317, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.29)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 2.30198783887
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.3019878388729937, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.30)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 2.84070308522
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.8407030852198814, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.84)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: 2.37763955113
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 2.377639551130833, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.38)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 0.401439299034
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 0.40143929903355013, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.40)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: forward, reward: 0.751546346325
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 0.7515463463245441, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.75)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 0.308204119899
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 0.3082041198987103, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.31)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: left, reward: 2.04919871986
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 2.0491987198557986, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.05)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: 1.7215546496
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 1.7215546496032637, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.72)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: right, reward: 0.00210306383459
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 0.002103063834594132, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.00)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: forward, reward: 1.51159949106
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': 1.511599491063849, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 1.51)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: left, reward: 0.814880190007
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 0.8148801900073064, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.81)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: -4.50515501523
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 16, 't': 14, 'action': None, 'reward': -4.50515501522723, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.51)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: -4.54791007526
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 15, 't': 15, 'action': None, 'reward': -4.547910075258583, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.55)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: forward, reward: -40.7194422389
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': -40.71944223888308, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.72)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 2.31918734753
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 13, 't': 17, 'action': None, 'reward': 2.3191873475269817, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.32)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 1.69898301565
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.6989830156482468, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.70)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: forward, reward: -9.86180936716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': -9.861809367157289, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.86)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: left, reward: -40.9089471092
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 10, 't': 20, 'action': 'left', 'reward': -40.90894710921816, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.91)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 0.307425645535
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 0.30742564553481155, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.31)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 0.504863877181
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 0.5048638771809979, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.50)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 2.28983571849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 23, 'action': None, 'reward': 2.2898357184938134, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.29)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: forward, reward: -39.4827657759
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 6, 't': 24, 'action': 'forward', 'reward': -39.482765775926, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.48)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 2.02360520881
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 2.023605208807301, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.02)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.77400838505
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 4, 't': 26, 'action': None, 'reward': 1.774008385047658, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.77)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: -39.7076018295
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', None), 'deadline': 3, 't': 27, 'action': 'left', 'reward': -39.707601829514395, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.71)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 0.531070819702
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 2, 't': 28, 'action': None, 'reward': 0.5310708197020853, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.53)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: 0.922022384927
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 1, 't': 29, 'action': 'left', 'reward': 0.9220223849272573, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.92)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 55
\-------------------------
Environment.reset(): Trial set up with start = (7, 3), destination = (4, 4), deadline = 20
Simulating trial. . .
epsilon = 0.5769; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: left, reward: -40.6612489042
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'left'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -40.6612489041997, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.66)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: -40.2181933228
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', 'left'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -40.2181933227555, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.22)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: left, reward: -9.3081084482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 18, 't': 2, 'action': 'left', 'reward': -9.308108448198272, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent attempted driving left through a red light. (rewarded -9.31)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 1.37443486505
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.374434865046008, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.37)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: left, reward: -10.7188779148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -10.71887791475145, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.72)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: forward, reward: 2.13025318855
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 2.1302531885457014, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 2.13)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: left, reward: -9.51515450395
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': -9.515154503951402, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.52)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 1.13335222033
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.1333522203273485, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 1.13)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 1.94202861388
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.942028613878941, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.94)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 1.8708433197
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.8708433196999463, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.87)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 2.37471967293
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.374719672933257, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.37)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: -4.6881439277
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': None, 'reward': -4.688143927704996, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.69)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: 1.5730502859
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'right'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.5730502858958368, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'right')
Agent drove right instead of left. (rewarded 1.57)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: 2.16381042082
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 2.163810420818053, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.16)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 2.43851492005
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.4385149200546277, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.44)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 1.51634836323
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.5163483632299117, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.52)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 0.495737381142
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.49573738114239774, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.50)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: left, reward: -0.634329196762
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': -0.6343291967619483, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove left instead of right. (rewarded -0.63)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: right, reward: -0.647440928977
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': -0.6474409289773944, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded -0.65)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 0.928805958736
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.9288059587356285, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 0.93)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 56
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (3, 2), deadline = 20
Simulating trial. . .
epsilon = 0.5712; alpha = 0.1000
Simulating trial. . .
epsilon = 0.5712; alpha = 0.1000
Simulating trial. . .
epsilon = 0.5712; alpha = 0.1000
Simulating trial. . .
epsilon = 0.5712; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: forward, reward: 1.14058927719
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'right'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.140589277194675, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'right')
Agent drove forward instead of left. (rewarded 1.14)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 2.64672841378
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.646728413776093, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.65)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 2.20331113404
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.2033111340389993, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.20)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 1.75515824662
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.755158246615791, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.76)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 2.56610265739
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.56610265738565, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.57)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: forward, reward: 0.0906059551086
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 0.09060595510862623, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove forward instead of left. (rewarded 0.09)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: right, reward: 0.260214218234
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.2602142182344298, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 0.26)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: -9.57570922679
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -9.575709226788543, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.58)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 1.40473349973
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.4047334997333396, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 1.40)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.54960160265
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.5496016026524853, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.55)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: right, reward: 1.68836089467
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'forward'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.688360894671829, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'forward')
Agent followed the waypoint right. (rewarded 1.69)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: right, reward: 0.64039798547
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.6403979854695097, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.64)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: None, reward: 1.86061882582
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.860618825824247, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.86)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: left, reward: 0.753673015954
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.7536730159535949, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.75)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: left, reward: 1.18764562805
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 1.187645628052183, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 1.19)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: right, reward: 2.06310689805
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 2.0631068980503553, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.06)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: 1.22496708089
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.2249670808890225, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.22)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 0.886384595828
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.8863845958275682, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 0.89)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 2.18739797316
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 2.1873979731605884, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.19)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.77862688623
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.7786268862308734, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.78)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 57
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (2, 7), deadline = 20
Simulating trial. . .
epsilon = 0.5655; alpha = 0.1000
Simulating trial. . .
epsilon = 0.5655; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 1.83592887764
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.835928877639564, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.84)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: -5.1570324998
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': -5.157032499801419, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.16)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 0.013583932152
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'right'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.013583932152039568, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'right')
Agent drove right instead of forward. (rewarded 0.01)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: left, reward: 1.41509452313
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 1.4150945231349663, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.42)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 2.00803459299
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.008034592985954, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.01)
75% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 58
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (7, 5), deadline = 25
Simulating trial. . .
epsilon = 0.5599; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 0.724477701343
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 0.7244777013430532, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.72)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 0.345283953593
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.3452839535927976, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'left')
Agent drove right instead of left. (rewarded 0.35)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 2.55779828727
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 2.5577982872746063, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.56)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: left, reward: 0.182763871974
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 0.18276387197355115, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove left instead of forward. (rewarded 0.18)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 1.91347077003
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.913470770034846, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.91)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 1.74751957501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.7475195750058736, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.75)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 1.91212127151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.9121212715081866, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.91)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: left, reward: 0.740677641038
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 0.7406776410376783, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.74)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: forward, reward: -39.5236398541
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -39.523639854107095, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.52)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 2.6974009111
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.697400911095425, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.70)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: forward, reward: -9.62365172115
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': -9.623651721149947, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.62)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 2.59534661405
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 2.5953466140546104, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.60)
52% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 59
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (1, 4), deadline = 25
Simulating trial. . .
epsilon = 0.5543; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 1.08782305387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.087823053872148, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.09)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 1.28340368704
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.2834036870381578, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.28)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 2.25276665823
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.252766658226093, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.25)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 2.35302801609
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.3530280160866908, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.35)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: -10.3456618583
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -10.345661858267265, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.35)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: forward, reward: 2.71470061051
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.714700610512776, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.71)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: -9.64369808618
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': -9.643698086184969, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.64)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: -4.42388083303
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 18, 't': 7, 'action': None, 'reward': -4.42388083303413, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.42)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: right, reward: 1.73338787783
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.73338787783154, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 1.73)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: left, reward: 1.99505645103
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.9950564510289024, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.00)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: -4.08499659723
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 15, 't': 10, 'action': None, 'reward': -4.084996597226212, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.08)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: left, reward: -9.87367002875
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': -9.873670028754631, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent attempted driving left through a red light. (rewarded -9.87)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 1.44111251709
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.4411125170900423, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.44)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 2.46049223852
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 12, 't': 13, 'action': None, 'reward': 2.460492238524744, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.46)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 1.2525552811
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.2525552811025769, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.25)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 0.642333712904
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 0.6423337129043016, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.64)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: left, reward: -40.3805731743
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 9, 't': 16, 'action': 'left', 'reward': -40.38057317428005, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.38)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 2.25534941158
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 17, 'action': None, 'reward': 2.2553494115785893, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.26)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.04084223873
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.0408422387313068, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.04)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: forward, reward: -9.97028124149
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': -9.970281241486084, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.97)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 2.21965766985
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 2.2196576698500814, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.22)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: 2.03389401376
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 2.0338940137557904, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.03)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 0.605534623369
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.6055346233690371, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent drove right instead of forward. (rewarded 0.61)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: 0.928158692693
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 23, 'action': 'left', 'reward': 0.9281586926931511, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.93)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: -0.897549920355
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': -0.8975499203552355, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded -0.90)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 60
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (6, 6), deadline = 25
Simulating trial. . .
epsilon = 0.5488; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: None, reward: 1.50674685453
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'right'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.5067468545262597, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'right')
Agent properly idled at a red light. (rewarded 1.51)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 0.51576752262
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.5157675226199441, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.52)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: left, reward: 0.273823199088
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 0.273823199088006, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove left instead of forward. (rewarded 0.27)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: right, reward: 1.93714104158
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.9371410415796222, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.94)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 2.24676779192
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.246767791915997, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.25)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: right, reward: 1.42416136221
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'right'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.4241613622135803, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'right')
Agent drove right instead of forward. (rewarded 1.42)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: forward, reward: 0.585663865965
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 0.5856638659650609, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.59)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: None, reward: 2.71654335878
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.7165433587812493, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.72)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: None, reward: 1.0604184085
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.0604184085004233, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.06)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: -0.0947631229208
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': -0.09476312292081523, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded -0.09)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 0.470124532229
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.47012453222930095, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.47)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 1.00900153998
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.0090015399805845, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.01)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: forward, reward: -40.9473119853
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'left'), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': -40.94731198534603, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.95)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 2.48848897378
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 2.488488973780305, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 2.49)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 1.85856730352
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.858567303522319, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.86)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: left, reward: 0.593523893188
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 0.5935238931879838, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.59)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: 0.783596470032
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'left'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 0.7835964700318635, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'left')
Agent drove forward instead of right. (rewarded 0.78)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: forward, reward: 1.46910449649
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 1.4691044964900994, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent drove forward instead of right. (rewarded 1.47)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: forward, reward: -0.259564095301
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', None), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': -0.25956409530086, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', None)
Agent drove forward instead of right. (rewarded -0.26)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 1.58919077258
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 6, 't': 19, 'action': None, 'reward': 1.5891907725815766, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.59)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 0.700989029295
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.7009890292949328, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.70)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: left, reward: -9.42178312169
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': -9.42178312168611, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent attempted driving left through a red light. (rewarded -9.42)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: forward, reward: 0.299064054829
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': 0.2990640548293617, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 0.30)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: right, reward: 1.13408831087
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 2, 't': 23, 'action': 'right', 'reward': 1.1340883108689803, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.13)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: right, reward: -0.498761588864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 1, 't': 24, 'action': 'right', 'reward': -0.49876158886381106, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded -0.50)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 61
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (6, 2), deadline = 20
Simulating trial. . .
epsilon = 0.5434; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: left, reward: -39.5260382313
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'right'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -39.52603823131902, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.53)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: -9.69556865267
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -9.695568652668008, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.70)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: -0.0134180001014
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': -0.013418000101429128, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded -0.01)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 1.2397360179
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.2397360179000418, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.24)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 1.40633407473
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.4063340747301698, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.41)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: -19.3177927886
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': -19.317792788558993, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.32)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 0.424069960453
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.42406996045303114, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 0.42)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 2.85938879473
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 2.8593887947260828, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.86)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: -4.99604809478
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 12, 't': 8, 'action': None, 'reward': -4.99604809478185, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.00)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: -10.5892815993
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': -10.589281599275054, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.59)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: left, reward: -10.8959789702
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -10.895978970160314, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent attempted driving left through a red light. (rewarded -10.90)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: left, reward: -9.53907898766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': -9.539078987664327, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.54)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 1.34224149189
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.3422414918895207, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.34)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: forward, reward: 0.389994164661
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.38999416466108194, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.39)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: left, reward: 0.75733995676
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 0.7573399567599245, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove left instead of right. (rewarded 0.76)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 2.08303398725
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.083033987249917, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.08)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 1.11313031042
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.1131303104197823, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.11)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: left, reward: -0.120525403631
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': -0.12052540363076891, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded -0.12)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 1.55587302399
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 1.5558730239929028, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.56)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: -10.7610477623
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -10.761047762265372, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.76)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 62
\-------------------------
Environment.reset(): Trial set up with start = (7, 6), destination = (4, 3), deadline = 30
Simulating trial. . .
epsilon = 0.5379; alpha = 0.1000
Simulating trial. . .
epsilon = 0.5379; alpha = 0.1000
Simulating trial. . .
epsilon = 0.5379; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 1.9608636201
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'right', 'forward'), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.9608636201012182, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.96)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 2.57200913126
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.5720091312639948, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.57)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: -40.7508976612
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': -40.75089766116192, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.75)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 2.3017162848
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 2.301716284797599, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.30)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: -5.37139700629
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'right'), 'deadline': 26, 't': 4, 'action': None, 'reward': -5.371397006289673, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.37)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: 2.63135180529
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 2.6313518052921023, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.63)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.02480890499
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.02480890498985, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.02)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.24353209387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 23, 't': 7, 'action': None, 'reward': 2.243532093872541, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.24)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: -9.14897563756
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': -9.148975637555194, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.15)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: 1.69155569421
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.6915556942128676, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.69)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: -4.542426423
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 20, 't': 10, 'action': None, 'reward': -4.542426423001487, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.54)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 1.65521471372
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.6552147137174305, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.66)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: -10.4922556608
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': 'left', 'reward': -10.49225566081648, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.49)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 1.39939775393
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 1.3993977539318176, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.40)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: left, reward: -9.01838430348
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': 'left', 'reward': -9.018384303476251, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.02)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 1.51323312369
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.5132331236888705, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.51)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 1.11609425131
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 1.116094251307267, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.12)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: right, reward: 1.8755647067
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 1.875564706699418, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.88)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: -9.31624871978
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 12, 't': 18, 'action': 'forward', 'reward': -9.31624871977719, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -9.32)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: -9.92541510988
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 11, 't': 19, 'action': 'left', 'reward': -9.925415109879014, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.93)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 0.834710790056
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 10, 't': 20, 'action': None, 'reward': 0.8347107900557555, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 0.83)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: 0.994413435429
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 0.9944134354293058, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.99)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: left, reward: 0.613538179106
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 8, 't': 22, 'action': 'left', 'reward': 0.6135381791057175, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 0.61)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: 1.2655059653
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 7, 't': 23, 'action': 'forward', 'reward': 1.265505965304222, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 1.27)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: -0.281798057393
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 6, 't': 24, 'action': 'right', 'reward': -0.2817980573932308, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded -0.28)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: -0.233305796441
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 5, 't': 25, 'action': None, 'reward': -0.2333057964414882, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.23)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 1.4885001809
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 4, 't': 26, 'action': 'right', 'reward': 1.4885001809039937, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 1.49)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: 2.04995186125
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 3, 't': 27, 'action': 'right', 'reward': 2.0499518612493715, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 2.05)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.76224753936
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 28, 'action': None, 'reward': 1.7622475393595243, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.76)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 0.399464202134
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 1, 't': 29, 'action': 'right', 'reward': 0.3994642021337931, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.40)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 63
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (6, 3), deadline = 20
Simulating trial. . .
epsilon = 0.5326; alpha = 0.1000
Simulating trial. . .
epsilon = 0.5326; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 1.10093776702
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.1009377670197282, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'forward')
Agent followed the waypoint right. (rewarded 1.10)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.13710798081
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.137107980807575, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.14)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.8823106186
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.882310618602019, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.88)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.70109771803
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.701097718025255, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.70)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 0.0117956115819
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.011795611581922327, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.01)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 1.49072958685
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'right'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.4907295868525599, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'right')
Agent drove right instead of left. (rewarded 1.49)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: forward, reward: -9.48240552139
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -9.48240552138817, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.48)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: forward, reward: 0.14807476675
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 0.14807476674977238, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove forward instead of left. (rewarded 0.15)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: right, reward: 0.233201701365
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.23320170136519003, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded 0.23)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: None, reward: 2.30174178885
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.3017417888471012, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.30)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: right, reward: 2.00654815197
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'right', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 2.0065481519663466, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', None)
Agent followed the waypoint right. (rewarded 2.01)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 0.375031674066
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.3750316740656354, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.38)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: None, reward: 2.07656573905
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.076565739049374, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', None)
Agent properly idled at a red light. (rewarded 2.08)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: left, reward: 1.9750018274
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.9750018273981313, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.98)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 1.27327813789
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.2732781378910145, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.27)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: left, reward: -9.80703884082
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -9.80703884081997, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.81)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: -0.269921249296
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 4, 't': 16, 'action': None, 'reward': -0.2699212492961154, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.27)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 0.0945697518397
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.09456975183968785, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent drove right instead of forward. (rewarded 0.09)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: left, reward: 0.853361334799
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 2, 't': 18, 'action': 'left', 'reward': 0.853361334798767, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.85)
5% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 64
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (3, 4), deadline = 20
Simulating trial. . .
epsilon = 0.5273; alpha = 0.1000
Simulating trial. . .
epsilon = 0.5273; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: forward, reward: -10.7027337038
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', 'right', 'right'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -10.702733703802854, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', 'right')
Agent attempted driving forward through a red light. (rewarded -10.70)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 2.79279138406
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.792791384059832, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.79)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 2.36303628093
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.3630362809332635, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.36)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.29558147129
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.2955814712922475, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.30)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: right, reward: 2.69837181255
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.698371812549994, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.70)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 1.17731763872
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.17731763871508, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'left')
Agent drove right instead of forward. (rewarded 1.18)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.64740090214
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.6474009021402523, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.65)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 2.38567300013
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.3856730001306143, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.39)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.77584610067
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.7758461006659723, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.78)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: left, reward: 1.04813616439
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.048136164392526, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.05)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: -10.5626716649
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': -10.562671664866647, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -10.56)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: 0.0681752691983
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.06817526919829064, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.07)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 1.09670896965
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.096708969653839, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.10)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.54138518114
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.5413851811393458, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.54)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: forward, reward: 0.49717973684
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 0.4971797368395864, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent drove forward instead of right. (rewarded 0.50)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: forward, reward: 0.758753958325
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 0.7587539583251972, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 0.76)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: left, reward: -19.581839923
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 4, 't': 16, 'action': 'left', 'reward': -19.581839922993087, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.58)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: left, reward: -10.6588372331
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': -10.658837233120643, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent attempted driving left through a red light. (rewarded -10.66)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: forward, reward: -10.8985837006
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': -10.898583700627531, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -10.90)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 1.49217014851
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.492170148505415, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 1.49)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 65
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (5, 4), deadline = 20
Simulating trial. . .
epsilon = 0.5220; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 2.3011689065
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'right'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.301168906495028, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'right')
Agent properly idled at a red light. (rewarded 2.30)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: -39.1556307363
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -39.155630736271696, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.16)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.82140276415
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.8214027641521366, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.82)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 2.02040651626
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.020406516260222, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.02)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 2.34004812665
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.3400481266461353, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.34)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: -5.385392515
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': -5.385392515003853, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.39)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 0.944767264984
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 0.9447672649838166, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove forward instead of left. (rewarded 0.94)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 2.81202274865
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.812022748650185, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 2.81)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: left, reward: 1.01759058182
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.0175905818210471, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.02)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: forward, reward: 0.34640049411
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.34640049411028695, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.35)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.48872358606
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.4887235860605195, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.49)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 2.07968235715
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 2.0796823571476257, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.08)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: -0.248299496365
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': -0.24829949636483817, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent drove right instead of forward. (rewarded -0.25)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 0.305971734968
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.30597173496780206, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.31)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 2.42832784261
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.428327842614477, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.43)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.63475640305
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.6347564030498347, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.63)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 1.88573715823
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.8857371582298041, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.89)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 1.19093720151
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 1.1909372015109376, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.19)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: left, reward: -9.81737065029
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -9.817370650294345, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.82)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: left, reward: -10.0710155546
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': -10.071015554612176, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.07)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 66
\-------------------------
Environment.reset(): Trial set up with start = (5, 5), destination = (8, 7), deadline = 25
Simulating trial. . .
epsilon = 0.5169; alpha = 0.1000
Simulating trial. . .
epsilon = 0.5169; alpha = 0.1000
Simulating trial. . .
epsilon = 0.5169; alpha = 0.1000
Simulating trial. . .
epsilon = 0.5169; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: left, reward: -10.5308382614
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': -10.530838261445265, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.53)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: 1.35520182726
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.355201827256589, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.36)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: right, reward: 0.13319999105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 0.13319999104981894, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.13)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 1.07297190976
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.072971909760766, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.07)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: forward, reward: -9.45421346287
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -9.454213462868298, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -9.45)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: 0.100544350278
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.10054435027775888, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 0.10)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: right, reward: 1.48654413384
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.4865441338437781, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent drove right instead of forward. (rewarded 1.49)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: 0.140542856823
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 0.14054285682279355, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded 0.14)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 1.56433266922
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 1.5643326692165007, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.56)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 2.41765567371
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.417655673706591, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.42)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: -40.1122774307
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'right'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': -40.11227743071661, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'right')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.11)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 0.818121095553
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 0.8181210955527309, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.82)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: left, reward: 0.588422262606
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 13, 't': 12, 'action': 'left', 'reward': 0.5884222626060005, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.59)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: right, reward: 1.39822824352
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'forward'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.398228243521064, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 1.40)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 1.28393711576
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 1.283937115763655, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 1.28)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: left, reward: 2.52227120575
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 2.5222712057508954, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.52)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: forward, reward: 0.79264636998
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 0.7926463699795395, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 0.79)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: forward, reward: -9.55231389149
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': -9.55231389148541, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent attempted driving forward through a red light. (rewarded -9.55)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: 1.22787597832
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.2278759783183686, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.23)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: -0.159124975081
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': -0.1591249750806053, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded -0.16)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 1.29022503224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 1.2902250322410445, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.29)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: left, reward: -40.6732013154
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': -40.67320131541101, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.67)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 1.60954912981
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 1.609549129806824, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.61)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 1.92320895384
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.9232089538382366, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.92)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 1.34450926477
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.344509264765574, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.34)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 67
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (8, 4), deadline = 30
Simulating trial. . .
epsilon = 0.5117; alpha = 0.1000
Simulating trial. . .
epsilon = 0.5117; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 1.5016865164
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'left'), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.501686516401121, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.50)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 2.72344600882
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.7234460088153574, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.72)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: left, reward: -10.8927669029
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 28, 't': 2, 'action': 'left', 'reward': -10.892766902885187, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent attempted driving left through a red light. (rewarded -10.89)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 2.83695657289
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.836956572888738, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.84)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: forward, reward: 1.15344456063
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 1.1534445606297807, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.15)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: left, reward: -40.6309368021
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': -40.63093680205287, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.63)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: left, reward: -10.261169903
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 24, 't': 6, 'action': 'left', 'reward': -10.261169902963722, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent attempted driving left through a red light. (rewarded -10.26)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: right, reward: 0.45174002897
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 0.45174002896979537, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove right instead of forward. (rewarded 0.45)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 0.373889420033
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 0.3738894200328996, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 0.37)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 2.09699200876
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 2.0969920087638085, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.10)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: 1.44781030168
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.4478103016783024, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.45)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 1.4384980017
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.4384980016954654, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.44)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: forward, reward: 1.51981557284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': 1.5198155728375098, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.52)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 1.80454016774
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 1.8045401677407198, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.80)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: -0.0840739166599
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', 'right'), 'deadline': 16, 't': 14, 'action': 'right', 'reward': -0.08407391665993913, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', 'right')
Agent drove right instead of left. (rewarded -0.08)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: left, reward: 0.665274205106
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 15, 't': 15, 'action': 'left', 'reward': 0.6652742051056125, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove left instead of right. (rewarded 0.67)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: 2.15009983457
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 2.1500998345687075, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.15)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: -9.24756135862
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': -9.247561358624209, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.25)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 0.871351444611
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', None), 'deadline': 12, 't': 18, 'action': None, 'reward': 0.8713514446111728, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', None)
Agent properly idled at a red light. (rewarded 0.87)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: 1.7790475023
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': 1.779047502296043, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.78)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 2.33781171594
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 2.3378117159442473, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.34)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 2.17721471505
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 9, 't': 21, 'action': None, 'reward': 2.177214715045502, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.18)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 1.01168426114
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 22, 'action': None, 'reward': 1.0116842611410763, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.01)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 1.16144070327
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 1.161440703273212, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.16)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: 1.84799761873
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 6, 't': 24, 'action': 'left', 'reward': 1.847997618730429, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.85)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 0.461692083942
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 0.461692083941746, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 0.46)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: forward, reward: -40.7028947254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'forward'), 'deadline': 4, 't': 26, 'action': 'forward', 'reward': -40.70289472540601, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.70)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 1.7202368835
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 3, 't': 27, 'action': 'right', 'reward': 1.7202368834964572, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 1.72)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: forward, reward: 0.441924801037
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 2, 't': 28, 'action': 'forward', 'reward': 0.441924801036646, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.44)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 0.448933101338
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 1, 't': 29, 'action': 'right', 'reward': 0.448933101338032, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.45)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 68
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (6, 2), deadline = 20
Simulating trial. . .
epsilon = 0.5066; alpha = 0.1000
Simulating trial. . .
epsilon = 0.5066; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: 1.4480686545
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.4480686544989532, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.45)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: forward, reward: -40.9363067413
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -40.93630674132037, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.94)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 1.77807848929
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.7780784892865036, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.78)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: right, reward: 1.01657788739
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.016577887389981, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.02)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 2.44161125777
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.4416112577713154, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.44)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: left, reward: 1.86498567547
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.864985675474038, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.86)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: right, reward: 1.03468040545
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.0346804054509973, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.03)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: left, reward: 1.60097773528
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.6009777352790087, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.60)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: 1.7386394613
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.7386394613044778, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.74)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: 1.66044709206
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.6604470920575722, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.66)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.75132483297
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.7513248329671227, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.75)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: right, reward: 2.14305972027
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 2.1430597202696804, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.14)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 1.24453220866
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 1.2445322086646176, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.24)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.93208933963
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.932089339627236, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.93)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 0.538035946668
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 0.5380359466682613, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.54)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: forward, reward: 1.40487981308
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 1.404879813079022, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent drove forward instead of right. (rewarded 1.40)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 1.60525862388
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.605258623880776, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.61)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 2.02575460098
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 2.0257546009750813, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.03)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: -5.85037088862
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 2, 't': 18, 'action': None, 'reward': -5.850370888623571, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.85)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 2.07548393034
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 2.0754839303389376, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 2.08)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 69
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (1, 6), deadline = 25
Simulating trial. . .
epsilon = 0.5016; alpha = 0.1000
Simulating trial. . .
epsilon = 0.5016; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 1.44420310189
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', 'right'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.444203101887684, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', 'right')
Agent properly idled at a red light. (rewarded 1.44)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 2.50999466797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.5099946679674763, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.51)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 1.92099479404
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.9209947940413352, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.92)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 2.04674087054
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.0467408705386143, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.05)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: -5.80933557125
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': -5.809335571245393, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.81)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: -5.67346143033
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': -5.6734614303329325, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.67)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: 1.53613633696
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.5361363369638583, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.54)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: -19.5266848782
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': -19.526684878234068, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.53)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: -10.8586228362
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -10.858622836151351, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.86)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: left, reward: 1.27155888359
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.2715588835913767, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove left instead of forward. (rewarded 1.27)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: forward, reward: 0.686202503432
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 0.6862025034315186, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', None)
Agent drove forward instead of right. (rewarded 0.69)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: 2.2314637905
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 2.231463790502513, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.23)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: -0.0410231997615
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': -0.041023199761502926, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent drove right instead of left. (rewarded -0.04)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: -19.3215513113
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'forward'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': -19.321551311294776, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.32)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: -10.0475503059
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': -10.047550305914811, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -10.05)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 1.26365383108
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.2636538310813819, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.26)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 2.57644581052
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 9, 't': 16, 'action': None, 'reward': 2.57644581051729, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.58)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 1.5581529285
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.558152928498007, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.56)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: 2.27806250154
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 2.278062501542696, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.28)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 1.118099716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 6, 't': 19, 'action': None, 'reward': 1.1180997159951502, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.12)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: -40.6817807921
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 5, 't': 20, 'action': 'forward', 'reward': -40.68178079212015, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.68)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 0.472680351742
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 0.4726803517416097, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.47)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 0.315843430797
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': 0.31584343079714006, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove forward instead of left. (rewarded 0.32)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: left, reward: 1.83593177248
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 2, 't': 23, 'action': 'left', 'reward': 1.835931772484444, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 1.84)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 1.49754210405
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.4975421040525663, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.50)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 70
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (6, 3), deadline = 25
Simulating trial. . .
epsilon = 0.4966; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4966; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4966; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4966; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: left, reward: -10.2127786292
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 25, 't': 0, 'action': 'left', 'reward': -10.212778629199402, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent attempted driving left through a red light. (rewarded -10.21)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: left, reward: -9.82413667162
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -9.824136671623332, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent attempted driving left through a red light. (rewarded -9.82)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: right, reward: 0.91038851942
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 0.9103885194196586, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 0.91)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.34218815923
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.3421881592255436, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.34)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.8297681149
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.8297681149011487, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.83)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: 1.05745229165
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.057452291649846, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.06)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: -19.7844523205
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': -19.784452320539824, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.78)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: -10.0816204589
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': -10.081620458908365, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.08)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: -9.29667460255
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -9.296674602550862, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent attempted driving forward through a red light. (rewarded -9.30)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: 0.997459319964
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'right'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.9974593199643711, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 1.00)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: -19.4492393163
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'right'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': -19.449239316275442, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'right')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.45)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 1.10100955056
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.1010095505553241, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.10)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 0.997538816375
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 13, 't': 12, 'action': None, 'reward': 0.9975388163750079, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.00)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: left, reward: 0.0670294584284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 0.06702945842836971, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.07)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 2.24731482334
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.2473148233384963, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.25)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: 1.13176799256
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.1317679925551132, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.13)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: right, reward: 1.37023093947
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 1.3702309394655718, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 1.37)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: left, reward: -9.66459834303
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': -9.664598343027366, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.66)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 1.20291261314
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.2029126131414571, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.20)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: left, reward: 0.660238128792
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 0.6602381287923909, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.66)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: left, reward: 0.727915591569
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 20, 'action': 'left', 'reward': 0.7279155915692916, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.73)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: forward, reward: 0.977026217367
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 0.9770262173673889, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 0.98)
12% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 71
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (3, 5), deadline = 20
Simulating trial. . .
epsilon = 0.4916; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: forward, reward: 1.7917195062
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.7917195061955697, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.79)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: left, reward: 1.86118994605
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.861189946054992, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.86)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: right, reward: 1.91537158607
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.9153715860734477, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 1.92)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 2.07742379666
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.0774237966612565, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.08)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: 2.48833624229
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.4883362422918793, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.49)
75% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 72
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (4, 6), deadline = 20
Simulating trial. . .
epsilon = 0.4868; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 1.82209234019
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.8220923401904687, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.82)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: -39.6055576727
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', 'left'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -39.60555767265359, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.61)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: -19.5930467541
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': -19.59304675412535, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.59)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 2.95177661517
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 2.951776615172994, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.95)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 1.17746844598
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.1774684459790627, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 1.18)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 2.26065322062
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.2606532206237557, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.26)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 0.95683698716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 0.956836987160103, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.96)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: 1.53068210054
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.5306821005408395, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.53)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 1.15138688022
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'right'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.151386880220724, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'right')
Agent followed the waypoint right. (rewarded 1.15)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 73
\-------------------------
Environment.reset(): Trial set up with start = (2, 6), destination = (6, 5), deadline = 25
Simulating trial. . .
epsilon = 0.4819; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4819; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: -10.7057207574
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -10.705720757405922, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.71)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 0.728212299106
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.7282122991059632, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent drove right instead of left. (rewarded 0.73)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.68466254849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.6846625484881554, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.68)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: forward, reward: -10.5302668545
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': -10.530266854493204, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.53)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: right, reward: 1.80909102806
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.8090910280567671, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 1.81)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: right, reward: 1.26644679498
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.2664467949843674, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 1.27)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.53593876467
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.5359387646652767, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.54)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: left, reward: 2.282192841
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 2.2821928410002466, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.28)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 0.655449301399
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.6554493013993706, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', None)
Agent drove right instead of forward. (rewarded 0.66)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: left, reward: -10.775727108
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'left', 'reward': -10.775727108024451, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.78)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 2.71475489264
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.714754892638626, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.71)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: forward, reward: -0.0652856712097
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': -0.06528567120973505, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded -0.07)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: None, reward: 2.42745941844
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'right'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.4274594184449354, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'right')
Agent properly idled at a red light. (rewarded 2.43)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: forward, reward: 0.61879082985
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 0.6187908298496307, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.62)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: left, reward: 2.15667732739
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 2.1566773273881585, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.16)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 1.12330628214
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.123306282135222, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.12)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: -9.52293378359
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': -9.522933783587339, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.52)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: forward, reward: 2.24199306977
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 2.241993069768853, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.24)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: forward, reward: -9.41130176878
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': -9.411301768782014, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.41)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: -0.256758647079
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 6, 't': 19, 'action': 'right', 'reward': -0.2567586470789781, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded -0.26)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: forward, reward: -39.9663902181
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 5, 't': 20, 'action': 'forward', 'reward': -39.966390218140205, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.97)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: forward, reward: 2.29072749087
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 2.2907274908677606, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.29)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 0.18130015935
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.18130015934983335, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.18)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: left, reward: 1.7616076162
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 2, 't': 23, 'action': 'left', 'reward': 1.7616076161980248, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.76)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 0.372902095764
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.37290209576351185, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.37)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 74
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (4, 4), deadline = 30
Simulating trial. . .
epsilon = 0.4771; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: right, reward: 0.617065270735
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 0.6170652707348998, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.62)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: 2.22669678801
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 29, 't': 1, 'action': 'left', 'reward': 2.226696788008975, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 2.23)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.40095527337
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.4009552733736768, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.40)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.5968545345
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.59685453450418, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.60)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: 1.47958684006
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 1.479586840063173, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.48)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: left, reward: 1.91152128076
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 1.9115212807574689, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 1.91)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: left, reward: 0.563808563751
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 0.5638085637505678, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent drove left instead of forward. (rewarded 0.56)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: 1.99900859069
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.999008590685845, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.00)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 2.11506716812
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.1150671681158326, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.12)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: left, reward: -9.7672221089
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 9, 'action': 'left', 'reward': -9.767222108904592, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.77)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: left, reward: 1.36915948975
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 1.3691594897515635, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 1.37)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: -4.76580464035
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': None, 'reward': -4.765804640352707, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.77)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: forward, reward: 0.0666944917639
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': 0.06669449176389908, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent drove forward instead of right. (rewarded 0.07)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: 0.49776581449
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'left', None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 0.4977658144900341, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'left', None)
Agent drove left instead of right. (rewarded 0.50)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 0.4461855239
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'right'), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 0.44618552389984023, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'right')
Agent drove right instead of left. (rewarded 0.45)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: forward, reward: -40.1185204394
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': -40.11852043935529, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.12)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: 1.32906719768
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 1.3290671976788653, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 1.33)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 2.50618302836
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 13, 't': 17, 'action': None, 'reward': 2.506183028356926, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.51)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: -10.6601059552
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 18, 'action': 'left', 'reward': -10.660105955222217, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.66)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 0.152773592278
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 0.1527735922775274, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.15)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: 1.82256226741
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 20, 'action': None, 'reward': 1.8225622674054383, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.82)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: 1.52950571564
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 1.5295057156358216, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.53)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 0.824489388465
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 22, 'action': None, 'reward': 0.8244893884647173, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.82)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: left, reward: 1.18579580559
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 23, 'action': 'left', 'reward': 1.1857958055891444, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.19)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: 0.761122476621
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 6, 't': 24, 'action': None, 'reward': 0.7611224766209939, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 0.76)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: -0.337466547964
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 5, 't': 25, 'action': 'forward', 'reward': -0.33746654796449227, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent drove forward instead of right. (rewarded -0.34)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: left, reward: -10.5960302722
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 4, 't': 26, 'action': 'left', 'reward': -10.596030272201647, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.60)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: -20.892710962
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 3, 't': 27, 'action': 'right', 'reward': -20.892710962010643, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.89)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: 1.36741277875
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 2, 't': 28, 'action': 'right', 'reward': 1.3674127787461323, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.37)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 1.65643572175
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 1, 't': 29, 'action': 'right', 'reward': 1.6564357217521772, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.66)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 75
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (2, 2), deadline = 25
Simulating trial. . .
epsilon = 0.4724; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: 1.70070415225
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.7007041522539585, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.70)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 2.87147157407
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.871471574070598, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.87)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 2.10367480621
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.103674806213943, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.10)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: -9.49085818402
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': -9.490858184024797, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.49)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: 1.2717983268
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.2717983268029722, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 1.27)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 1.21713854071
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.217138540709856, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.22)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: right, reward: 0.359952400776
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.3599524007760221, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.36)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: forward, reward: 0.425829270182
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 0.4258292701816453, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.43)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: 1.07607769844
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.0760776984436133, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.08)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: 1.79792537988
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.7979253798755888, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.80)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: -5.43668772435
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': -5.436687724352755, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.44)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.74673887737
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.7467388773675605, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.75)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: -39.2531392414
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': -39.25313924138567, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.25)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: 0.176634291963
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 0.176634291962515, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent drove right instead of forward. (rewarded 0.18)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: 1.60433296691
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 1.6043329669080721, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded 1.60)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 2.52847136731
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.528471367311524, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.53)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 2.39642483006
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 2.3964248300622293, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.40)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: left, reward: -40.1127604915
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': -40.11276049147561, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.11)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: right, reward: 2.17991081716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 2.1799108171615478, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.18)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: left, reward: 1.38387447996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 1.3838744799641889, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 1.38)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 0.739142961873
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 0.7391429618730248, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.74)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: -0.289373511032
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': None, 'reward': -0.28937351103226583, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.29)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: left, reward: 0.937096348952
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 3, 't': 22, 'action': 'left', 'reward': 0.9370963489521802, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.94)
8% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 76
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (2, 5), deadline = 25
Simulating trial. . .
epsilon = 0.4677; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4677; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4677; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4677; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4677; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4677; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.86438426898
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.864384268982186, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.86)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.38713365137
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'right'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.387133651374805, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 2.39)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 1.33959264447
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.3395926444675363, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.34)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: -5.59526582839
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': -5.5952658283875785, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.60)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 1.20648483907
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.2064848390676057, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.21)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: -5.58493027302
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': -5.584930273017453, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.58)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 2.15474267099
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.154742670989191, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.15)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: -10.2684969339
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': -10.268496933876598, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.27)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.73785069875
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.737850698750539, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.74)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.52938170309
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.5293817030940424, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.53)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 2.13383528947
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 2.133835289469487, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.13)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 2.59209058088
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.5920905808844705, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.59)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 1.25848321357
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.258483213570674, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.26)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 0.632771536604
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 0.6327715366040403, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 0.63)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.33445312428
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.3344531242847864, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.33)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: right, reward: 1.34868188995
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.348681889954279, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.35)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 0.765164471499
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 0.7651644714992204, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded 0.77)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.50845568937
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'right'), 'deadline': 8, 't': 17, 'action': None, 'reward': 1.5084556893656396, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.51)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 2.0151169609
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 2.015116960900693, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.02)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: forward, reward: 0.255400108293
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 0.25540010829309934, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 0.26)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: left, reward: 0.807476093811
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 5, 't': 20, 'action': 'left', 'reward': 0.8074760938106973, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent drove left instead of right. (rewarded 0.81)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 0.956907081661
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 0.9569070816606464, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.96)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 0.757949806675
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.7579498066752628, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 0.76)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.50706261938
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.5070626193820769, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.51)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: -0.484469742503
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 1, 't': 24, 'action': 'right', 'reward': -0.48446974250310904, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded -0.48)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 77
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (8, 3), deadline = 25
Simulating trial. . .
epsilon = 0.4630; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: right, reward: -19.2301178495
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': -19.230117849485275, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.23)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: -40.7932266663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -40.793226666326625, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.79)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 1.57077937483
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'right'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.5707793748290548, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'right')
Agent properly idled at a red light. (rewarded 1.57)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 1.99131129651
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.9913112965088617, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.99)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 0.158138896343
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 0.158138896343048, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent drove right instead of left. (rewarded 0.16)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 1.60546154769
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.6054615476944323, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 1.61)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: 2.37626705504
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.3762670550430833, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 2.38)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: -4.12437125941
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': -4.124371259408415, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.12)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: -5.88113886989
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 17, 't': 8, 'action': None, 'reward': -5.88113886988924, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.88)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 0.99195015645
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 0.991950156450282, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 0.99)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 1.26767985109
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.267679851087639, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.27)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: left, reward: 0.483383380707
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 0.4833833807071325, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.48)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 0.996701055152
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 0.9967010551522388, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.00)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.6564453309
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.6564453309048335, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.66)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.52756256475
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.5275625647453985, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.53)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.98981722805
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.9898172280451378, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.99)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 0.833560621704
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 9, 't': 16, 'action': None, 'reward': 0.8335606217042952, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.83)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 1.54212906119
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 1.5421290611948657, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.54)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 2.5248556031
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': 2.5248556030958706, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.52)
24% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 78
\-------------------------
Environment.reset(): Trial set up with start = (7, 3), destination = (2, 5), deadline = 25
Simulating trial. . .
epsilon = 0.4584; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4584; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4584; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4584; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: forward, reward: -39.2455368078
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -39.24553680777096, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.25)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: forward, reward: -10.8608330934
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': -10.860833093446196, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.86)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 0.282390720349
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 0.28239072034864776, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 0.28)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: left, reward: 1.74898938082
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 1.748989380816661, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.75)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: left, reward: -40.5429905598
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'right'), 'deadline': 21, 't': 4, 'action': 'left', 'reward': -40.54299055979386, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.54)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 0.557844033275
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.5578440332754696, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent drove right instead of left. (rewarded 0.56)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: 2.01709896418
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.0170989641825567, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', None)
Agent followed the waypoint forward. (rewarded 2.02)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: left, reward: -0.0780965502373
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': -0.07809655023726902, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded -0.08)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: 1.2222913615
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.2222913615046964, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.22)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 1.01815319967
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.0181531996699202, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.02)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 0.841120345269
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 0.8411203452687295, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.84)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 0.433089620342
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 14, 't': 11, 'action': None, 'reward': 0.43308962034189713, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.43)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: -0.0929939007049
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', 'left'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': -0.09299390070490254, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', 'left')
Agent drove right instead of forward. (rewarded -0.09)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: 0.898186017168
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 0.8981860171678351, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 0.90)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 2.44020585244
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.440205852441636, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.44)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 2.59104916416
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.591049164159796, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.59)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: left, reward: 1.89396892678
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 1.8939689267782802, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.89)
32% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 79
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (6, 4), deadline = 25
Simulating trial. . .
epsilon = 0.4538; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4538; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: forward, reward: -39.8726359491
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -39.87263594911734, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.87)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: forward, reward: -9.92497117607
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': -9.924971176066995, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.92)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 1.80831430405
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.8083143040530758, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.81)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 2.87171930524
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.871719305240494, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.87)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.19002528164
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.1900252816366264, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.19)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: left, reward: 0.751113590414
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 0.7511135904141485, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.75)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: -5.70960323233
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': -5.709603232327182, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.71)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 2.89303967837
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 2.8930396783691417, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.89)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 2.53472257193
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.534722571929345, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.53)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 2.27558230288
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.2755823028757334, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.28)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: left, reward: 1.05860260617
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 1.0586026061690352, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.06)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 80
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (1, 5), deadline = 20
Simulating trial. . .
epsilon = 0.4493; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4493; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4493; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 2.80116651528
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.801166515278024, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 2.80)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 1.16105508281
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.161055082812097, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 1.16)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: 0.366912389565
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.36691238956509464, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 0.37)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: 1.87264775097
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 1.8726477509714763, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.87)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 1.64777334214
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.6477733421428047, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 1.65)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: forward, reward: 1.21538000643
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.2153800064345364, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.22)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: 1.81279360709
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.8127936070932458, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.81)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.74622339008
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.7462233900777508, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.75)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: left, reward: -39.9285438141
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'left'), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -39.92854381409685, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.93)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: left, reward: 1.11377644956
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.1137764495640112, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.11)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: -19.7744788604
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': -19.774478860384534, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.77)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 2.50285162602
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'right'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 2.5028516260194427, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'right')
Agent followed the waypoint right. (rewarded 2.50)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 1.17188933882
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.1718893388225524, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.17)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: left, reward: 0.956507933441
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.9565079334406145, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove left instead of forward. (rewarded 0.96)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.606641773
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.6066417730043199, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 1.61)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: -0.0761989218905
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': -0.07619892189045152, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove right instead of forward. (rewarded -0.08)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: -5.34981750288
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': -5.349817502881081, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.35)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 2.31632409255
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 3, 't': 17, 'action': None, 'reward': 2.3163240925526005, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.32)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: left, reward: -9.53690611877
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -9.536906118772222, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.54)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 0.623958249909
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.6239582499085731, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.62)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 81
\-------------------------
Environment.reset(): Trial set up with start = (2, 2), destination = (8, 4), deadline = 20
Simulating trial. . .
epsilon = 0.4449; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4449; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4449; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 1.42930166215
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.429301662148727, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.43)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: left, reward: -9.47701151248
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'right', 'forward'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -9.477011512482777, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'forward')
Agent attempted driving left through a red light. (rewarded -9.48)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 1.90715973088
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.9071597308847927, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'forward')
Agent followed the waypoint right. (rewarded 1.91)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 1.09455384031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.0945538403093344, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.09)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: -5.18735596823
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': -5.187355968229346, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.19)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 1.30427637624
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.3042763762449996, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.30)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 2.07906337889
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.079063378885585, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.08)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: 2.56717660316
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 2.5671766031636345, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.57)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.02037178335
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.0203717833501893, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.02)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 2.11163934111
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.111639341105124, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.11)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 0.444549071916
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.44454907191637305, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent drove right instead of left. (rewarded 0.44)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 1.17275882106
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.172758821061662, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.17)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 2.54756103119
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.5475610311870636, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.55)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 1.3562466785
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.3562466785025098, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.36)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 0.944719888567
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.9447198885667756, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.94)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 1.00367561433
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.0036756143306826, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 1.00)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 0.991569337378
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'forward'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 0.991569337377664, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.99)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: left, reward: 0.449598587665
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 0.44959858766528793, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 0.45)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 2.04494623032
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', 'forward'), 'deadline': 2, 't': 18, 'action': None, 'reward': 2.044946230321245, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.04)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: left, reward: -10.797526958
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 1, 't': 19, 'action': 'left', 'reward': -10.797526958013808, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.80)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 82
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (6, 6), deadline = 25
Simulating trial. . .
epsilon = 0.4404; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4404; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4404; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: left, reward: 1.0862101549
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 1.0862101548955436, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.09)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 2.15850923805
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.1585092380468645, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.16)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: right, reward: 2.35018517263
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 2.3501851726269676, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.35)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 1.44654688097
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.4465468809690276, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.45)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: 2.59874605293
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.5987460529294184, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.60)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: -4.90011293306
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': -4.900112933057153, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.90)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: 1.30520639822
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.3052063982209416, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent drove right instead of left. (rewarded 1.31)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: forward, reward: -10.2423392881
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': -10.242339288131522, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -10.24)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: 1.76899319432
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.7689931943219386, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.77)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: left, reward: -39.4052476052
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 16, 't': 9, 'action': 'left', 'reward': -39.40524760520533, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.41)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: 0.0108656677358
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 0.010865667735753815, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 0.01)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: forward, reward: -9.32247304477
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': -9.322473044766877, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.32)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: right, reward: 1.0304421546
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.0304421545991422, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.03)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: forward, reward: -0.00544152691764
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'forward'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': -0.005441526917643991, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'forward')
Agent drove forward instead of right. (rewarded -0.01)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: 0.767293671946
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 0.7672936719458335, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove left instead of right. (rewarded 0.77)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 1.92474305061
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.924743050611686, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.92)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 1.67364018304
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.673640183037833, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.67)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 0.792648221076
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 0.7926482210758956, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 0.79)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 0.624141669595
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 0.6241416695945539, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.62)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: left, reward: -10.0418039215
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 6, 't': 19, 'action': 'left', 'reward': -10.041803921548125, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.04)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 2.05145483368
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 2.051454833678301, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.05)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 0.704375596286
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': None, 'reward': 0.7043755962856749, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.70)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 1.13018488114
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 1.1301848811399398, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove right instead of forward. (rewarded 1.13)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: left, reward: -9.89870762634
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 2, 't': 23, 'action': 'left', 'reward': -9.898707626339323, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -9.90)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 0.871814716295
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.8718147162947534, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.87)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 83
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (7, 4), deadline = 25
Simulating trial. . .
epsilon = 0.4360; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: left, reward: 1.4631600898
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'left'), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 1.4631600898026695, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'left')
Agent followed the waypoint left. (rewarded 1.46)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 2.04277137956
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.0427713795649844, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.04)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 1.03019629197
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.0301962919746286, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.03)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 2.05110963317
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.0511096331700203, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.05)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: forward, reward: -40.2614698555
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -40.26146985554195, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.26)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: 1.28338479206
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.2833847920575217, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.28)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 0.301995249554
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.30199524955401713, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent drove right instead of forward. (rewarded 0.30)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: forward, reward: 0.158106492382
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 0.15810649238186003, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.16)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: 1.25106050941
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 1.2510605094122338, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.25)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.85199652865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.8519965286499105, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.85)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: 0.834588810986
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.8345888109863138, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.83)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 1.90047420263
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.900474202632544, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.90)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 0.884569630423
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 0.8845696304230155, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.88)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: left, reward: 1.52662211186
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 1.5266221118561794, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 1.53)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: -0.233823981378
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': -0.2338239813775861, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded -0.23)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: -9.97783941073
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 10, 't': 15, 'action': 'left', 'reward': -9.97783941073162, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.98)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: 1.1364125678
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 1.1364125678026733, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 1.14)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: -5.94114934549
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 8, 't': 17, 'action': None, 'reward': -5.941149345493692, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.94)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 1.4694249674
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 1.4694249673994353, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 1.47)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 1.65905773079
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 19, 'action': None, 'reward': 1.659057730791, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.66)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 0.413862505787
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.41386250578738526, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.41)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: -0.452733126995
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 4, 't': 21, 'action': 'right', 'reward': -0.4527331269951552, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded -0.45)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 0.819795706825
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 3, 't': 22, 'action': None, 'reward': 0.8197957068254684, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.82)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: forward, reward: 0.644654617927
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 2, 't': 23, 'action': 'forward', 'reward': 0.6446546179268047, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.64)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 1.91187911306
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.9118791130578667, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.91)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 84
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (7, 3), deadline = 20
Simulating trial. . .
epsilon = 0.4317; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.77882015055
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.7788201505494765, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.78)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 1.57103122746
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'right'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.5710312274588465, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'right')
Agent drove right instead of forward. (rewarded 1.57)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 1.24110100509
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.2411010050912903, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.24)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 2.48894564884
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.4889456488355624, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.49)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: 1.83335524094
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.8333552409376623, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 1.83)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 2.2753389816
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.2753389816038356, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.28)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: 0.893437455563
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 0.8934374555628866, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.89)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: forward, reward: 0.551602157145
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 0.5516021571445306, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent drove forward instead of right. (rewarded 0.55)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 1.89820310227
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.8982031022704726, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.90)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: right, reward: 1.57310984301
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.5731098430105734, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 1.57)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 1.12017878704
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.1201787870369344, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent drove right instead of left. (rewarded 1.12)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 0.734417314653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 0.7344173146532553, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 0.73)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 1.24580007457
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.24580007457446, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.25)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 0.689744907213
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.6897449072130822, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent drove right instead of left. (rewarded 0.69)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: left, reward: 1.04473307553
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 1.0447330755291828, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent drove left instead of right. (rewarded 1.04)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: forward, reward: 0.599651351672
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 0.5996513516723696, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent followed the waypoint forward. (rewarded 0.60)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: forward, reward: 1.26189929056
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.26189929055921, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.26)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: right, reward: -0.661331330488
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 3, 't': 17, 'action': 'right', 'reward': -0.6613313304882266, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent drove right instead of forward. (rewarded -0.66)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: left, reward: -10.1007365613
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -10.10073656134699, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent attempted driving left through a red light. (rewarded -10.10)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: left, reward: 1.96358537132
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 1.96358537131836, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.96)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 85
\-------------------------
Environment.reset(): Trial set up with start = (2, 7), destination = (7, 2), deadline = 20
Simulating trial. . .
epsilon = 0.4274; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 2.5894698858
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'right'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.589469885797995, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'right')
Agent followed the waypoint forward. (rewarded 2.59)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 2.96315416675
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 2.963154166751143, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.96)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.73547390335
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.735473903352816, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.74)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.05725602096
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.0572560209648123, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.06)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: -9.57176524698
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -9.571765246982554, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.57)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.43714894288
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.4371489428800561, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.44)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: left, reward: 1.46288252635
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.462882526352551, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 1.46)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: left, reward: 0.595468001235
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 0.5954680012346554, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.60)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 0.980990383357
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.9809903833570022, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.98)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: left, reward: 1.04907591007
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.0490759100660625, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 1.05)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: 1.41096560009
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.4109656000854969, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.41)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: -4.93634227249
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': -4.936342272494316, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.94)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: left, reward: 1.51158128944
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 1.5115812894369272, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.51)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 1.13561624102
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.135616241021311, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.14)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 2.28882328259
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.2888232825914026, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.29)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: 1.85557018719
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 1.8555701871915973, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.86)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 1.54844002501
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.548440025008793, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.55)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.17486357986
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 2.174863579856487, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.17)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 0.84131254651
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.84131254650976, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 0.84)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 0.920518455152
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.9205184551522481, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 0.92)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 86
\-------------------------
Environment.reset(): Trial set up with start = (7, 4), destination = (1, 6), deadline = 20
Simulating trial. . .
epsilon = 0.4232; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4232; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4232; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4232; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 1.48892019361
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.488920193612355, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.49)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: -9.69932401815
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -9.699324018154963, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.70)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 2.43140156412
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.431401564122859, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.43)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 1.73782509343
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.7378250934321258, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.74)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: left, reward: 1.85661645745
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.8566164574471282, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.86)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: forward, reward: -10.5383444542
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': -10.538344454246001, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.54)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: 1.00627413807
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.0062741380713542, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.01)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: forward, reward: 0.347911518302
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 0.3479115183018626, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.35)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: forward, reward: 1.66700642299
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.6670064229944401, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 1.67)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 1.2873427195
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.2873427195032205, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.29)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 1.11354169537
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.1135416953658694, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.11)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 0.431771693782
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.43177169378229396, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.43)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.28505419168
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.285054191678351, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.29)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 0.100119487405
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': None, 'reward': 0.10011948740542487, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.10)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: 1.59635482803
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 1.596354828027065, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.60)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: -0.113154701572
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': -0.11315470157239371, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded -0.11)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 0.907134048241
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'right'), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.907134048241385, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 0.91)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: left, reward: -39.6141038687
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': -39.61410386869862, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.61)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: -19.3804489359
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': -19.38044893592934, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.38)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 0.575868904053
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.5758689040530425, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.58)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 87
\-------------------------
Environment.reset(): Trial set up with start = (4, 7), destination = (6, 5), deadline = 20
Simulating trial. . .
epsilon = 0.4190; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4190; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4190; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 1.68737189534
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'right'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.6873718953378938, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'right')
Agent drove right instead of left. (rewarded 1.69)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.85239271531
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.8523927153091009, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.85)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: -10.8554053375
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': -10.855405337532389, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.86)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.73274790015
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.7327479001508506, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.73)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 1.50814036244
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.5081403624354603, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.51)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.5147188849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.514718884898747, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.51)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 1.00176233788
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.0017623378815474, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.00)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 2.01101206512
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.0110120651195937, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.01)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: forward, reward: -10.878935146
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -10.878935145970338, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.88)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: left, reward: 0.821465526992
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 0.8214655269921625, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.82)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: left, reward: 1.0056248326
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.005624832601069, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove left instead of right. (rewarded 1.01)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 2.67401250283
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 2.674012502832693, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.67)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 2.0077150285
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.007715028503524, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.01)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: -0.204508371218
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': None, 'reward': -0.20450837121807008, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.20)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: 0.505687448353
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 0.505687448352781, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove left instead of right. (rewarded 0.51)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.09443881986
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.0944388198638952, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.09)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 2.19241115699
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 2.192411156993632, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.19)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 1.20317515496
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 1.2031751549639562, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 1.20)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: left, reward: -10.3920731021
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -10.392073102144746, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.39)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: left, reward: 2.06903671996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 2.0690367199557, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.07)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 88
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (8, 4), deadline = 30
Simulating trial. . .
epsilon = 0.4148; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: right, reward: 1.14561208201
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'right', 'right'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.1456120820057731, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'right', 'right')
Agent followed the waypoint right. (rewarded 1.15)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.81233729157
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.8123372915747087, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.81)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: -10.5137314712
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 28, 't': 2, 'action': 'left', 'reward': -10.51373147122246, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.51)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.88034997046
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.880349970462046, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.88)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: -9.40791473211
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': -9.407914732114833, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.41)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.71592308516
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.71592308516212, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.72)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: forward, reward: 2.70848917158
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 2.7084891715764483, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.71)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: right, reward: 1.23645450125
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.2364545012458321, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent drove right instead of forward. (rewarded 1.24)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 1.27803616476
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 1.2780361647592486, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.28)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 1.71744092733
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 1.7174409273269475, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 1.72)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: right, reward: 1.13306189498
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.133061894976353, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.13)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 0.953250814061
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 19, 't': 11, 'action': None, 'reward': 0.9532508140608003, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.95)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 2.71462523084
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.714625230839906, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.71)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: right, reward: 0.553483336516
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 0.5534833365159777, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.55)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: left, reward: 1.16022397398
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 16, 't': 14, 'action': 'left', 'reward': 1.160223973976182, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.16)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 2.04442515253
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': 2.0444251525322015, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.04)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 1.89222043012
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 1.8922204301232501, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.89)
43% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 89
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (1, 5), deadline = 20
Simulating trial. . .
epsilon = 0.4107; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4107; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 1.30696077491
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.3069607749073406, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.31)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 2.72186633488
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 2.721866334876803, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.72)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 2.43055181461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.430551814606963, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.43)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: -40.3555400322
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': -40.35554003215287, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.36)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.28142942471
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.2814294247127875, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.28)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 2.31773320226
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.3177332022603205, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.32)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: -4.14917278482
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 14, 't': 6, 'action': None, 'reward': -4.149172784824281, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.15)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 0.781699178063
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.7816991780631657, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.78)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: left, reward: 1.17917293014
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.1791729301401053, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent followed the waypoint left. (rewarded 1.18)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.21633161253
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.2163316125312635, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.22)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.75540437075
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.7554043707466693, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.76)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.35804718031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.358047180312365, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.36)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.25470221465
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.2547022146548241, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.25)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: left, reward: 0.757368903411
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.7573689034112217, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 0.76)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: left, reward: 0.938350153957
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 0.9383501539567812, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', None)
Agent drove left instead of forward. (rewarded 0.94)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 1.93538725704
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.9353872570395094, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.94)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: 0.58732391751
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 0.5873239175103657, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.59)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 2.303356007
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 2.303356007000594, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.30)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: left, reward: 0.599734983267
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': 0.5997349832673375, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent drove left instead of right. (rewarded 0.60)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 0.128351395935
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.1283513959350855, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 0.13)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 90
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (6, 6), deadline = 20
Simulating trial. . .
epsilon = 0.4066; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: right, reward: 0.487177282574
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.4871772825737326, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove right instead of left. (rewarded 0.49)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: forward, reward: 1.20001898099
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.2000189809866604, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 1.20)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: forward, reward: 0.398074285169
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 0.39807428516868215, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.40)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 2.87757434774
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.8775743477407234, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.88)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 2.1092604168
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.1092604168027944, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.11)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: 1.40530653113
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.405306531130813, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 1.41)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: -9.64781772009
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -9.647817720090439, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.65)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: left, reward: 1.23399480299
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.2339948029947911, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.23)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 91
\-------------------------
Environment.reset(): Trial set up with start = (7, 4), destination = (5, 2), deadline = 20
Simulating trial. . .
epsilon = 0.4025; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4025; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4025; alpha = 0.1000
Simulating trial. . .
epsilon = 0.4025; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: left, reward: -10.0234294957
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -10.023429495719064, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -10.02)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 1.03490170813
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.034901708128317, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.03)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: right, reward: 2.27773942899
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.277739428987096, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.28)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 2.40834231618
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.4083423161829502, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.41)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 2.91165689788
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.911656897876969, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.91)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 1.51369666941
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.513696669406072, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.51)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: left, reward: 0.742458397774
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 0.74245839777435, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.74)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: right, reward: 1.2460451517
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.2460451516960276, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.25)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: forward, reward: -10.8804670711
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -10.880467071087166, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -10.88)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: left, reward: -19.9559422213
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': -19.95594222129293, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.96)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: left, reward: 2.41718502147
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 2.417185021469141, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.42)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: -4.79949152209
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': -4.799491522087167, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.80)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: -5.99585843837
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', None), 'deadline': 8, 't': 12, 'action': None, 'reward': -5.99585843837004, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -6.00)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: right, reward: 1.1028363882
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.1028363881973529, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.10)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.03770379698
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.037703796978531, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.04)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 1.15590767056
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.155907670560896, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.16)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 1.92318601342
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.923186013416925, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.92)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: -0.335907584891
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -0.33590758489133943, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove forward instead of left. (rewarded -0.34)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: -0.599597790741
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': -0.5995977907413939, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded -0.60)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: -9.12108127155
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 1, 't': 19, 'action': 'left', 'reward': -9.121081271546569, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.12)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 92
\-------------------------
Environment.reset(): Trial set up with start = (7, 2), destination = (2, 7), deadline = 20
Simulating trial. . .
epsilon = 0.3985; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: 1.47871170132
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.4787117013202078, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.48)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: left, reward: 0.331475842634
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 0.3314758426335521, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent drove left instead of right. (rewarded 0.33)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: 1.80289962469
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.8028996246929916, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.80)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: right, reward: 0.700519364089
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 0.7005193640890247, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 0.70)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 1.25934116827
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.2593411682739135, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.26)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 1.7869006558
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.7869006557956155, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.79)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: left, reward: 1.1093719987
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.109371998702741, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.11)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: left, reward: -20.5241519828
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': -20.524151982750304, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.52)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: -4.12763565899
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': -4.127635658992367, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.13)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 2.48628478462
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.4862847846210907, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.49)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 0.376180137378
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.3761801373775372, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.38)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: forward, reward: -40.4908284508
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': -40.49082845077202, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.49)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.8374868804
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.8374868803998774, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.84)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.96856057762
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.9685605776244195, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.97)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 0.748404188142
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 0.7484041881417051, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 0.75)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 0.412501197331
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 0.4125011973312489, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 0.41)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 0.765653090959
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.7656530909587842, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.77)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: -0.197338699815
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': -0.1973386998149348, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded -0.20)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: -4.93772719386
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 2, 't': 18, 'action': None, 'reward': -4.9377271938634335, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.94)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: 1.42170974808
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 1.4217097480816547, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.42)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 93
\-------------------------
Environment.reset(): Trial set up with start = (5, 7), destination = (2, 3), deadline = 25
Simulating trial. . .
epsilon = 0.3946; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3946; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3946; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 2.36953793357
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.369537933569779, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.37)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: forward, reward: -9.24029243014
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': -9.240292430142656, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent attempted driving forward through a red light. (rewarded -9.24)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: left, reward: -9.74195894141
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': -9.741958941408019, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -9.74)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: right, reward: 0.420870207824
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 0.42087020782411355, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 0.42)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 2.01730421117
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.017304211173739, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.02)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: left, reward: 0.689167641675
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', 'left'), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 0.6891676416751463, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', 'left')
Agent drove left instead of forward. (rewarded 0.69)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: None, reward: 1.54309096633
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.5430909663308934, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.54)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 0.973566970083
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.9735669700832643, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.97)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: -10.3588025928
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -10.358802592764642, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -10.36)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: 2.19584022881
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.1958402288087884, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.20)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: left, reward: 1.12088490105
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 1.1208849010512183, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove left instead of forward. (rewarded 1.12)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: left, reward: 1.13195099225
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.1319509922515687, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 1.13)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: left, reward: -9.74935575673
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': 'left', 'reward': -9.749355756731147, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.75)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.1562742093
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.1562742093025458, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.16)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.14852237874
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.1485223787412817, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.15)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.01639344611
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.0163934461138726, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.02)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: 0.277125538348
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 0.2771255383484016, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.28)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: right, reward: 1.96577367119
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'right'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.9657736711902898, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'right')
Agent followed the waypoint right. (rewarded 1.97)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: right, reward: 1.1247438357
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 1.1247438356981996, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.12)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: left, reward: 0.517108104345
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 0.5171081043447715, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent drove left instead of forward. (rewarded 0.52)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: right, reward: 1.99869666446
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 1.998696664458623, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.00)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 0.921676800894
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 0.9216768008943723, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.92)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: -40.9064110913
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 3, 't': 22, 'action': 'left', 'reward': -40.906411091318475, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.91)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 0.826633972283
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 23, 'action': None, 'reward': 0.8266339722826939, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.83)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.02488818147
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.0248881814731479, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.02)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 94
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (4, 6), deadline = 20
Simulating trial. . .
epsilon = 0.3906; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 2.27921324298
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.279213242984949, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.28)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 1.40292259895
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.4029225989454592, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.40)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: -4.82779269109
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 18, 't': 2, 'action': None, 'reward': -4.827792691085647, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.83)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 2.45285234227
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 2.4528523422683977, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 2.45)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 1.79037958111
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.7903795811126653, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.79)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 1.65600823411
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.6560082341059277, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.66)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 2.10587598835
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.105875988346536, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.11)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: forward, reward: 1.21927364886
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.219273648858874, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.22)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: left, reward: -19.3260107861
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -19.326010786129274, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.33)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: right, reward: 1.74472873066
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.7447287306560482, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.74)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: left, reward: -39.7561863168
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -39.75618631677051, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.76)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 1.19666333098
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.1966633309765644, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.20)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 2.01070877169
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.0107087716861267, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.01)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: 0.719439371502
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.7194393715021555, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.72)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: left, reward: 1.21921078266
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 1.2192107826635696, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.22)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: forward, reward: -0.390010523489
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -0.3900105234885556, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded -0.39)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: left, reward: 0.83939543135
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 0.8393954313504819, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 0.84)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: -40.92145025
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -40.92145024995224, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.92)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 0.97692712298
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 0.9769271229800516, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.98)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 1.8588117863
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.8588117863030762, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.86)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 95
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (7, 6), deadline = 25
Simulating trial. . .
epsilon = 0.3867; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3867; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 1.53659363561
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.536593635607956, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove right instead of left. (rewarded 1.54)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 1.92083340683
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'right'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.9208334068341892, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 1.92)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: right, reward: 1.6822126803
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.6822126803041237, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.68)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: forward, reward: 2.50721925679
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.507219256789104, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.51)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: 2.71822460807
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.718224608065599, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.72)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: left, reward: 1.47254072861
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.4725407286103818, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.47)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: forward, reward: -10.2881259374
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -10.288125937383171, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -10.29)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: -4.1535877161
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': -4.153587716102545, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.15)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: 1.2986290289
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 1.2986290288957578, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.30)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.7867969228
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.7867969228024614, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.79)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.04101883515
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.0410188351538743, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.04)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 2.0745001604
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 2.074500160402872, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 2.07)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 2.4805238602
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 2.480523860200347, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.48)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 0.900318586941
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 0.9003185869409478, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.90)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.23697329812
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.2369732981183215, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.24)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.18664518566
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.186645185658171, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 2.19)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: -0.257360338497
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': -0.2573603384967319, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded -0.26)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 1.36831098391
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.3683109839095418, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 1.37)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: forward, reward: 0.14207933037
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': 0.1420793303704393, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent drove forward instead of right. (rewarded 0.14)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: left, reward: -9.10044932395
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 6, 't': 19, 'action': 'left', 'reward': -9.100449323947839, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.10)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: forward, reward: -10.5289452403
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 5, 't': 20, 'action': 'forward', 'reward': -10.528945240321148, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.53)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 1.42259918588
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 1.4225991858849456, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.42)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 1.42617082949
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 1.4261708294868327, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.43)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: forward, reward: 0.420386179649
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 2, 't': 23, 'action': 'forward', 'reward': 0.42038617964902825, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.42)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: forward, reward: 0.242487589892
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': 0.24248758989238128, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 0.24)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 96
\-------------------------
Environment.reset(): Trial set up with start = (5, 7), destination = (7, 3), deadline = 20
Simulating trial. . .
epsilon = 0.3829; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 2.05522906781
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.0552290678116583, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.06)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 2.71230771026
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.7123077102645965, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.71)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 2.18342634224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.183426342243041, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.18)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 2.73313622801
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.733136228009995, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.73)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: left, reward: 2.18398218428
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 2.1839821842842424, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.18)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.6308958661
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.6308958661041952, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.63)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 0.511841576698
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'right'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.5118415766980895, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'right')
Agent drove right instead of forward. (rewarded 0.51)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 0.947694140306
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.9476941403060233, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.95)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: left, reward: 2.34121376959
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 2.3412137695873865, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 2.34)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 1.44802886011
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.448028860111118, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.45)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 1.23691475323
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.2369147532340452, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.24)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 2.04825423161
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.048254231612515, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.05)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 1.2728000676
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.272800067597699, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.27)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 0.945882481878
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': None, 'reward': 0.9458824818782167, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.95)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: 1.64133089032
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 1.6413308903216253, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.64)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 1.52632675628
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.5263267562804355, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.53)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: right, reward: 1.36965396413
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.3696539641287206, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 1.37)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: -5.37324885679
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': -5.373248856787336, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.37)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: forward, reward: 0.657879500152
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 0.6578795001516493, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent drove forward instead of left. (rewarded 0.66)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: 1.6650984235
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.6650984235049744, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'right', 'forward', None)
Agent properly idled at a red light. (rewarded 1.67)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 97
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (1, 6), deadline = 20
Simulating trial. . .
epsilon = 0.3791; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3791; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3791; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3791; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 1.76046615707
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.7604661570676658, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.76)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 0.784732912754
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 0.7847329127536561, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 0.78)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: -9.83446452131
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': -9.834464521310503, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.83)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 1.5988321822
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.5988321822003382, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.60)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 2.73325138576
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.7332513857554446, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.73)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.10995459929
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.1099545992878426, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.11)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 0.685114122738
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.6851141227375968, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.69)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: forward, reward: -9.36997973808
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -9.369979738083199, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.37)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: forward, reward: -10.2819123129
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -10.281912312869144, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -10.28)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 0.0145282730244
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.014528273024431826, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.01)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: -10.4301735461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': -10.430173546130346, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.43)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: -9.77807801114
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': -9.77807801113832, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent attempted driving forward through a red light. (rewarded -9.78)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.76048994136
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.7604899413588828, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.76)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 0.833585990499
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.8335859904986417, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 0.83)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 0.794234202659
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 0.7942342026586856, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove right instead of left. (rewarded 0.79)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 1.65257140753
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.6525714075300515, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.65)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 1.11585478325
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.1158547832495442, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.12)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: left, reward: 1.84120002837
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 1.841200028366431, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.84)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: left, reward: -0.274299673538
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -0.27429967353839224, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded -0.27)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: left, reward: -9.50989660141
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': -9.509896601405579, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.51)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 98
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (6, 4), deadline = 20
Simulating trial. . .
epsilon = 0.3753; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 2.20966364932
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.2096636493174526, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.21)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: -10.5899066633
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -10.589906663265678, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.59)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: -10.632053506
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': -10.632053505973845, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.63)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 1.35291690923
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.3529169092286173, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.35)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: 2.30868600784
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.308686007838239, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.31)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.96483650259
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.9648365025883352, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.96)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: -10.6350834079
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -10.635083407896396, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -10.64)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 1.42640070948
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.4264007094784015, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.43)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 1.2937890033
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.2937890033028152, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.29)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: left, reward: 1.53508244246
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.5350824424643568, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.54)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: left, reward: 1.09209910883
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.0920991088301781, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.09)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: right, reward: 2.54673612029
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 2.5467361202856322, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.55)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 99
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (5, 6), deadline = 25
Simulating trial. . .
epsilon = 0.3716; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3716; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3716; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3716; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 2.02720373256
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.0272037325602894, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.03)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: left, reward: -20.1858396973
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'right'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -20.185839697258146, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.19)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: 2.07110069771
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 2.0711006977128377, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.07)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 1.03112459508
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.0311245950815673, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.03)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: right, reward: 0.0672613963067
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 0.06726139630665717, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 0.07)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 0.980521955189
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 20, 't': 5, 'action': None, 'reward': 0.9805219551893187, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.98)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: forward, reward: -9.82072434157
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -9.82072434156958, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent attempted driving forward through a red light. (rewarded -9.82)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: forward, reward: 2.42831087208
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 2.428310872079706, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.43)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 1.59688016952
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.5968801695205879, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.60)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: forward, reward: 1.39433810985
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.3943381098488363, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.39)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 100
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (5, 2), deadline = 30
Simulating trial. . .
epsilon = 0.3679; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.55495054999
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.5549505499878855, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.55)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 2.40001409511
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 2.40001409511187, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 2.40)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 0.998455126817
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 0.9984551268165907, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.00)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.01199968953
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.0119996895320502, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.01)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.94956416578
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.9495641657799987, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.95)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 1.50578642278
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.5057864227794386, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent drove right instead of forward. (rewarded 1.51)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: -19.0024267163
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': -19.002426716259723, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.00)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 1.02236832375
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'right'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.0223683237531205, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'right')
Agent drove right instead of left. (rewarded 1.02)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.68134126986
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 1.6813412698625192, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.68)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: forward, reward: 0.0361582518402
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 0.036158251840191724, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 0.04)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 2.211131621
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 2.2111316210029446, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.21)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: -10.9768592514
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': -10.976859251410945, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.98)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 0.816082705844
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', 'forward'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 0.8160827058440171, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', 'forward')
Agent drove right instead of forward. (rewarded 0.82)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: 2.73271595923
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 2.7327159592335835, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.73)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 2.57270338236
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': 2.572703382362768, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.57)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: 2.17399653018
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': 2.173996530180556, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.17)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: -19.7543881087
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'left'), 'deadline': 14, 't': 16, 'action': 'right', 'reward': -19.754388108746248, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.75)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.40123776094
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.4012377609434725, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.40)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.35293470593
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.3529347059308097, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.35)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: 2.00486047781
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 11, 't': 19, 'action': 'left', 'reward': 2.0048604778057206, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.00)
33% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 101
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (3, 6), deadline = 25
Simulating trial. . .
epsilon = 0.3642; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 1.65954041294
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.6595404129402538, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.66)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: left, reward: 1.58388153529
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 1.5838815352947764, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', None)
Agent drove left instead of forward. (rewarded 1.58)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: right, reward: 2.28099534645
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 2.2809953464532864, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.28)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 2.37634489021
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.3763448902118522, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.38)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.21045808733
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.210458087329249, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.21)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.44419824541
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.4441982454142268, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.44)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.25357816903
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.253578169028625, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.25)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: -4.95382115582
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 18, 't': 7, 'action': None, 'reward': -4.953821155822527, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.95)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: 0.749747518772
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.7497475187719999, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 0.75)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: left, reward: 2.63243837205
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 2.632438372051781, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.63)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 2.80875688066
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.808756880662286, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.81)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: 2.09317969549
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 2.0931796954885, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.09)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: -5.97592593396
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 13, 't': 12, 'action': None, 'reward': -5.975925933964361, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.98)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 2.26446644269
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 12, 't': 13, 'action': None, 'reward': 2.264466442692229, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.26)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 1.0449093297
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'right'), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.0449093297033416, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.04)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: 0.403556561849
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 0.4035565618488881, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent drove forward instead of right. (rewarded 0.40)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: -10.6180208218
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': -10.618020821816302, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.62)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: right, reward: 1.08584069484
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'forward'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.0858406948372827, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 1.09)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 2.38338013052
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 2.383380130519406, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.38)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: 1.10469238756
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 1.1046923875571015, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.10)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: left, reward: 1.42561636808
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 5, 't': 20, 'action': 'left', 'reward': 1.425616368079085, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 1.43)
16% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 102
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (3, 5), deadline = 20
Simulating trial. . .
epsilon = 0.3606; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 1.59553369493
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.5955336949291512, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.60)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 1.78965432792
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.789654327920286, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.79)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: 1.50376651606
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.5037665160579408, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.50)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: 1.33324847486
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.3332484748604003, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.33)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 1.63206884587
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.6320688458662698, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.63)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 2.21046161708
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.2104616170769393, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.21)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 2.35896519056
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.3589651905648186, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.36)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: left, reward: -9.39389021483
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': -9.39389021482601, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.39)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 2.1096468932
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.1096468932033954, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.11)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: right, reward: 0.404248598618
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.40424859861820606, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 0.40)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: right, reward: 0.902018853178
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.9020188531778408, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.90)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 1.64082384921
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.6408238492078178, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.64)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: forward, reward: 0.0491943606777
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 0.04919436067768479, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove forward instead of right. (rewarded 0.05)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 1.33126586253
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.3312658625333913, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.33)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 0.958421788648
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.9584217886476358, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.96)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: right, reward: 1.92853054455
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.9285305445454641, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.93)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: 0.892426960344
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 0.892426960343683, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.89)
15% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 103
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (4, 2), deadline = 25
Simulating trial. . .
epsilon = 0.3570; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 1.37098994397
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.3709899439698443, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.37)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 1.56949806885
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.5694980688533116, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.57)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 2.83259411226
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.8325941122649043, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.83)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 1.92167030222
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.9216703022185992, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.92)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 2.09999683878
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.0999968387823413, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.10)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 2.16814878477
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.1681487847654513, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.17)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 0.727521268501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.7275212685008855, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent drove right instead of left. (rewarded 0.73)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 2.01435132274
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.0143513227400147, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.01)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 0.925016050784
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 0.9250160507840437, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.93)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: forward, reward: 0.995041284687
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 0.9950412846869932, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 1.00)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: forward, reward: 2.3004937175
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'left'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 2.3004937175020808, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'left')
Agent followed the waypoint forward. (rewarded 2.30)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 104
\-------------------------
Environment.reset(): Trial set up with start = (4, 3), destination = (2, 5), deadline = 20
Simulating trial. . .
epsilon = 0.3535; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3535; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3535; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: right, reward: 2.4292167176
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.4292167175985604, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 2.43)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 0.687445938674
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.687445938674312, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.69)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: -4.65879150708
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': -4.658791507077705, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.66)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 1.74059179258
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.740591792579567, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.74)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: left, reward: -9.66147207827
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -9.661472078267776, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.66)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: -4.87749713059
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': -4.8774971305874, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.88)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: right, reward: 1.05508265863
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.0550826586290003, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove right instead of left. (rewarded 1.06)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: 1.81247019163
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.8124701916299069, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.81)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: 2.20759368797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.2075936879672735, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.21)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: 2.52622008989
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.5262200898893097, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.53)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: left, reward: 1.63127989257
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.631279892567399, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 1.63)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: left, reward: 2.17934826113
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 2.1793482611277923, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.18)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 0.821226557706
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.8212265577062718, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 0.82)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.01066123107
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': None, 'reward': 2.0106612310728833, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.01)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 0.560287365765
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.5602873657650529, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', 'forward')
Agent properly idled at a red light. (rewarded 0.56)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: -19.9537593616
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': -19.953759361555143, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.95)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: -5.27546542846
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': None, 'reward': -5.275465428460489, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.28)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: -0.243304329619
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -0.24330432961926907, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded -0.24)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: 2.18861974165
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': 2.188619741651685, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.19)
5% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 105
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (5, 6), deadline = 35
Simulating trial. . .
epsilon = 0.3499; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 1.7661503824
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', 'right'), 'deadline': 35, 't': 0, 'action': 'forward', 'reward': 1.766150382401709, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', 'right')
Agent drove forward instead of left. (rewarded 1.77)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 2.03440648474
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'forward'), 'deadline': 34, 't': 1, 'action': 'forward', 'reward': 2.0344064847385823, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 2.03)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 2.63498323377
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 33, 't': 2, 'action': 'forward', 'reward': 2.634983233768281, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.63)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 1.74979482653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 32, 't': 3, 'action': None, 'reward': 1.749794826534054, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.75)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 1.59608428749
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 31, 't': 4, 'action': 'right', 'reward': 1.5960842874862609, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.60)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: left, reward: -40.8023484947
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 30, 't': 5, 'action': 'left', 'reward': -40.802348494738, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.80)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 1.48343787088
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 29, 't': 6, 'action': None, 'reward': 1.4834378708780507, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.48)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 2.80181901956
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 28, 't': 7, 'action': None, 'reward': 2.801819019562381, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.80)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: left, reward: -40.821965452
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 27, 't': 8, 'action': 'left', 'reward': -40.821965452032806, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.82)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: forward, reward: -0.0375996388944
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 26, 't': 9, 'action': 'forward', 'reward': -0.037599638894433296, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded -0.04)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: forward, reward: -39.2672737176
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'right'), 'deadline': 25, 't': 10, 'action': 'forward', 'reward': -39.26727371757641, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'right')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.27)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 2.35750025845
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 24, 't': 11, 'action': None, 'reward': 2.357500258447045, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.36)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: left, reward: -10.4723902344
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 23, 't': 12, 'action': 'left', 'reward': -10.472390234408678, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.47)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: -5.97823197203
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 22, 't': 13, 'action': None, 'reward': -5.978231972031757, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.98)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: 0.584524099771
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 21, 't': 14, 'action': 'right', 'reward': 0.5845240997712348, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 0.58)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 1.80639043491
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 20, 't': 15, 'action': None, 'reward': 1.806390434908929, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.81)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: -4.76875277571
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 19, 't': 16, 'action': None, 'reward': -4.768752775710282, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.77)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: left, reward: 1.24504410864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 18, 't': 17, 'action': 'left', 'reward': 1.2450441086375685, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.25)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 1.16450293061
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 18, 'action': None, 'reward': 1.1645029306137404, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.16)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 2.53843814831
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 19, 'action': None, 'reward': 2.5384381483054623, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.54)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: -20.1563964766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 15, 't': 20, 'action': 'right', 'reward': -20.15639647659235, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.16)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: left, reward: 0.978498261711
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 21, 'action': 'left', 'reward': 0.9784982617107241, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.98)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: -0.0645363705555
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 13, 't': 22, 'action': 'right', 'reward': -0.06453637055553774, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded -0.06)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.28307062927
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 12, 't': 23, 'action': None, 'reward': 1.2830706292711538, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.28)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: -9.8228671002
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 24, 'action': 'forward', 'reward': -9.822867100195474, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.82)
29% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: left, reward: 2.02398542448
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 25, 'action': 'left', 'reward': 2.023985424483797, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.02)
26% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: right, reward: 1.09896278826
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'right'), 'deadline': 9, 't': 26, 'action': 'right', 'reward': 1.098962788257944, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'right')
Agent drove right instead of left. (rewarded 1.10)
23% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: 1.8674775441
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 27, 'action': None, 'reward': 1.867477544100154, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.87)
20% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: right, reward: 2.29864813829
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 7, 't': 28, 'action': 'right', 'reward': 2.2986481382914454, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.30)
17% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: forward, reward: 0.394222787746
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 6, 't': 29, 'action': 'forward', 'reward': 0.39422278774565866, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent drove forward instead of right. (rewarded 0.39)
14% of time remaining to reach destination.
/-------------------
| Step 30 Results
\-------------------
Environment.step(): t = 30
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: right, reward: 0.988134450588
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 5, 't': 30, 'action': 'right', 'reward': 0.9881344505877045, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 0.99)
11% of time remaining to reach destination.
/-------------------
| Step 31 Results
\-------------------
Environment.step(): t = 31
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 1.21300727608
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 4, 't': 31, 'action': 'right', 'reward': 1.2130072760768438, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.21)
9% of time remaining to reach destination.
/-------------------
| Step 32 Results
\-------------------
Environment.step(): t = 32
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: forward, reward: 2.14194772161
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 32, 'action': 'forward', 'reward': 2.141947721608301, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.14)
6% of time remaining to reach destination.
/-------------------
| Step 33 Results
\-------------------
Environment.step(): t = 33
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: left, reward: 1.64077101581
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 33, 'action': 'left', 'reward': 1.6407710158065563, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.64)
3% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 106
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (1, 5), deadline = 25
Simulating trial. . .
epsilon = 0.3465; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: -20.4461185851
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': -20.44611858514239, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.45)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.32843102573
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.328431025732651, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.33)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.39573812542
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.3957381254214605, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.40)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.89117586142
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.8911758614226306, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.89)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: 1.04638331758
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 1.0463833175812056, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.05)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 1.55196930434
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.5519693043417846, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.55)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: -9.02396331899
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -9.023963318991436, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.02)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 1.36904146394
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.369041463935807, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.37)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 1.47875683731
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.4787568373116897, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.48)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: -5.62957235866
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': -5.629572358662855, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.63)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: left, reward: 1.67136974863
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 1.6713697486296994, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.67)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 1.88108370491
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.881083704912825, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.88)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: 0.0748441762747
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 13, 't': 12, 'action': 'left', 'reward': 0.07484417627472717, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove left instead of forward. (rewarded 0.07)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 1.0409505093
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.0409505092978268, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 1.04)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 0.345146912419
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 0.34514691241942674, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.35)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: left, reward: -9.95780817244
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': -9.957808172436392, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.96)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 0.764070783182
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 0.76407078318199, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.76)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: left, reward: 1.43492810958
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 1.4349281095758746, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.43)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: 1.68259999403
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 18, 'action': 'left', 'reward': 1.6825999940269678, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.68)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: -9.62868422849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 6, 't': 19, 'action': 'left', 'reward': -9.628684228487575, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -9.63)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: -10.9716645534
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 5, 't': 20, 'action': 'left', 'reward': -10.971664553410584, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -10.97)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.77260078195
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 4, 't': 21, 'action': None, 'reward': 1.772600781945161, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.77)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 1.11754110404
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 1.1175411040371603, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.12)
8% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 107
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (7, 5), deadline = 20
Simulating trial. . .
epsilon = 0.3430; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3430; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3430; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3430; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: forward, reward: 1.23290045245
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.2329004524515796, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 1.23)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: 0.298500054691
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'right'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 0.29850005469071894, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'right')
Agent drove forward instead of right. (rewarded 0.30)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: -20.9797165326
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': -20.979716532553095, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.98)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: left, reward: -39.1182583907
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -39.11825839066684, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.12)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 1.36436164511
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.3643616451148826, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 1.36)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 0.742872248767
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 0.7428722487671628, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent drove left instead of right. (rewarded 0.74)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 1.36143378161
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.361433781612705, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.36)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: -9.82109556402
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -9.821095564016058, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.82)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.12786945101
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.1278694510077245, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.13)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 2.79546371005
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.7954637100465947, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.80)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.23307483148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.2330748314781568, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.23)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 2.38767979891
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 2.387679798909537, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.39)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 1.8121851443
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 1.8121851443013308, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.81)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: left, reward: 0.304115522932
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.3041155229315603, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent drove left instead of forward. (rewarded 0.30)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 2.10778969241
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.107789692414289, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.11)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 0.711656535705
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 0.7116565357052504, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 0.71)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 108
\-------------------------
Environment.reset(): Trial set up with start = (7, 4), destination = (3, 5), deadline = 25
Simulating trial. . .
epsilon = 0.3396; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3396; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 0.576764682778
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 0.5767646827780304, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent drove right instead of left. (rewarded 0.58)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: 1.58816914117
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.5881691411730712, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', None)
Agent drove right instead of forward. (rewarded 1.59)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: left, reward: -39.904227489
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 23, 't': 2, 'action': 'left', 'reward': -39.9042274890231, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.90)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: 1.21072660609
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.2107266060893098, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.21)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: forward, reward: 1.29416877219
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 1.2941687721928594, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove forward instead of left. (rewarded 1.29)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: forward, reward: 0.0921535935662
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 0.09215359356623554, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent drove forward instead of left. (rewarded 0.09)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: left, reward: 2.28103946893
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 2.281039468925835, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.28)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.474716797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.4747167970010624, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.47)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: right, reward: 0.678578478059
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.6785784780594849, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.68)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: left, reward: 2.28250950664
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 2.2825095066365666, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.28)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: 1.46465021289
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 1.4646502128930985, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.46)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 2.17868642979
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 2.178686429786161, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.18)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.38165967724
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.3816596772439063, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.38)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.81447221501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.8144722150107824, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.81)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.66386494453
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.6638649445262599, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.66)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.46356307838
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.4635630783794455, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.46)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: -5.92805443886
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': -5.928054438861782, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.93)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 2.06230949039
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 2.0623094903936225, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.06)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 1.1769139139
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 1.1769139138963907, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', None)
Agent drove right instead of forward. (rewarded 1.18)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 2.03549951758
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 6, 't': 19, 'action': None, 'reward': 2.0354995175835917, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.04)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 0.273874018474
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'right'), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 0.27387401847447834, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'right')
Agent drove right instead of left. (rewarded 0.27)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: -20.5881801817
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 4, 't': 21, 'action': 'right', 'reward': -20.58818018166086, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.59)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 2.26279627314
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 3, 't': 22, 'action': None, 'reward': 2.2627962731392506, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.26)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: -9.16638399275
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 2, 't': 23, 'action': 'forward', 'reward': -9.166383992750655, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.17)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 0.79298546914
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 1, 't': 24, 'action': 'right', 'reward': 0.7929854691399956, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.79)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 109
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (3, 3), deadline = 20
Simulating trial. . .
epsilon = 0.3362; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3362; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3362; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3362; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: forward, reward: -10.6130957907
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', 'right', 'left'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -10.613095790716484, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'right', 'left')
Agent attempted driving forward through a red light. (rewarded -10.61)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.57541226677
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.5754122667654074, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.58)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: right, reward: 0.261051023869
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.26105102386939005, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 0.26)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: 2.58758623727
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.58758623726755, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.59)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: 2.54571054546
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.545710545459402, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.55)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: -5.1374648683
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': -5.137464868300352, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.14)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 1.04857561836
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.0485756183603678, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.05)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 1.25280116028
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.2528011602804405, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.25)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: left, reward: -9.42910356724
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -9.429103567236579, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.43)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 2.43426308781
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 2.434263087812493, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.43)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 1.0395979382
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.0395979381974474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.04)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 2.34735729022
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.347357290219712, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.35)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 2.13285481337
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'right'), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.1328548133746175, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'right')
Agent properly idled at a red light. (rewarded 2.13)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: -5.08442033791
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'right'), 'deadline': 7, 't': 13, 'action': None, 'reward': -5.084420337913083, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.08)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: left, reward: 1.47611886505
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 1.476118865048769, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove left instead of forward. (rewarded 1.48)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: 0.654550096767
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 0.6545500967670681, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 0.65)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 0.413466617872
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.4134666178723343, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.41)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: left, reward: 0.577139830749
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 0.5771398307485025, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.58)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 1.51657640121
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.5165764012111573, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.52)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: 1.46493464672
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 1.4649346467223026, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.46)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 110
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (6, 5), deadline = 25
Simulating trial. . .
epsilon = 0.3329; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 0.304957511176
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 0.3049575111762204, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 0.30)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 0.523522386824
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', 'forward'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.5235223868235419, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.52)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: left, reward: 2.71718697212
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 2.717186972117979, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.72)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: 2.01979623048
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.0197962304772448, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.02)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 2.70223937015
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.702239370148532, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.70)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.49079553026
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.4907955302587954, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.49)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.75560503453
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.7556050345346557, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.76)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 0.769646062381
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 0.7696460623810881, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.77)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: left, reward: -19.7240449618
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'forward'), 'deadline': 17, 't': 8, 'action': 'left', 'reward': -19.724044961795702, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.72)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.21029983667
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.210299836667308, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.21)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.42317607238
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.4231760723816933, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.42)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: 2.28166317367
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 2.281663173668474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.28)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 2.32299873945
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 2.3229987394494955, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.32)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: forward, reward: 2.39760216957
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 2.3976021695655367, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.40)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: None, reward: 2.22732677233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.227326772329535, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.23)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: left, reward: 1.61695946854
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 1.6169594685359856, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.62)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: right, reward: 1.96050328648
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 1.9605032864843963, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 1.96)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: right, reward: 2.24821395616
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 2.248213956160133, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.25)
28% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 111
\-------------------------
Environment.reset(): Trial set up with start = (2, 7), destination = (8, 3), deadline = 20
Simulating trial. . .
epsilon = 0.3296; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: forward, reward: -10.4326251096
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -10.432625109617604, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -10.43)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: left, reward: -40.8760282673
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'forward'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -40.87602826731835, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.88)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: right, reward: 1.31568788739
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.3156878873857056, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.32)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 1.29465948821
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.294659488213604, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.29)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.10413635597
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.1041363559654678, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.10)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 2.67547364676
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.6754736467617075, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.68)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 0.00687720501376
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.006877205013757992, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.01)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 2.23204054073
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 2.232040540727005, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.23)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.26326086544
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.263260865440367, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 1.26)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 0.99199297207
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.9919929720698251, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.99)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: left, reward: 1.2654340168
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.2654340167987468, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.27)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: forward, reward: 0.726733478064
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.7267334780635821, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.73)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 112
\-------------------------
Environment.reset(): Trial set up with start = (2, 6), destination = (6, 2), deadline = 30
Simulating trial. . .
epsilon = 0.3263; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 2.85309775261
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.8530977526124177, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.85)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 2.16270619094
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 2.1627061909441676, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.16)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: forward, reward: -9.76347970576
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': -9.763479705761283, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.76)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: right, reward: 1.87767929745
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 1.877679297447942, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.88)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 1.98167163865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.9816716386527136, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.98)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 1.08318063196
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.0831806319567003, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.08)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: left, reward: 1.39112433341
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 1.3911243334110845, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.39)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: forward, reward: 1.02242082715
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 1.0224208271452655, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.02)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 1.74807896322
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.748078963219898, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.75)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 2.73030674475
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 2.730306744750877, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.73)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: forward, reward: 2.36358909559
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 2.363589095586537, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.36)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 2.09481705599
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 2.0948170559911583, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 2.09)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 113
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (5, 5), deadline = 25
Simulating trial. . .
epsilon = 0.3230; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: left, reward: -10.9505093513
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': -10.950509351293253, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.95)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.10008403689
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.1000840368949396, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.10)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.27828346588
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.278283465878202, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.28)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.5940176167
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.5940176167048037, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.59)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: left, reward: -39.053888991
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'right'), 'deadline': 21, 't': 4, 'action': 'left', 'reward': -39.05388899101493, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.05)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: -10.2938234706
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'right'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': -10.293823470627665, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'right')
Agent attempted driving forward through a red light. (rewarded -10.29)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: left, reward: -20.7664662417
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': -20.76646624167217, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.77)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 1.16626781628
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.1662678162832631, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.17)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 2.27080592875
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.27080592874809, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.27)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 1.57942282817
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.5794228281684353, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.58)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: -5.69624845973
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': -5.696248459727383, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.70)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 1.59681276508
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.5968127650832002, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.60)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 1.85103394374
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.851033943739742, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.85)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 1.47444995228
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.4744499522800973, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.47)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 2.44042792957
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 2.4404279295735583, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.44)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 1.90845268567
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.908452685668459, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.91)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: 1.17032386937
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 1.17032386937033, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.17)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: left, reward: -40.9441916525
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'left'), 'deadline': 8, 't': 17, 'action': 'left', 'reward': -40.94419165246809, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.94)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 1.48893890577
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.4889389057717863, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.49)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: forward, reward: 0.786279051142
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 0.7862790511417941, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.79)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: left, reward: 1.70352839403
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 5, 't': 20, 'action': 'left', 'reward': 1.7035283940251376, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.70)
16% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 114
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (5, 3), deadline = 30
Simulating trial. . .
epsilon = 0.3198; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3198; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: left, reward: -10.6450829494
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 30, 't': 0, 'action': 'left', 'reward': -10.64508294942415, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent attempted driving left through a red light. (rewarded -10.65)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 1.02787169079
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.0278716907910859, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.03)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 1.6277565667
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.6277565666984626, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.63)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 2.03335808627
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 2.0333580862696805, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.03)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 2.36943007671
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.369430076708828, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.37)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.54473665113
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.5447366511332286, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.54)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: -5.94013555269
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': -5.940135552690655, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.94)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 2.02294064
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 2.0229406399990904, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.02)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 1.56446479762
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 1.5644647976191175, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.56)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: forward, reward: -40.6656840386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': -40.665684038594286, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.67)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 2.09110839524
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.0911083952431584, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.09)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 2.44524529289
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.445245292892203, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.45)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 1.10558374003
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.1055837400253818, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.11)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 1.16757358577
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 1.1675735857683833, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 1.17)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: left, reward: 1.26554986445
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 16, 't': 14, 'action': 'left', 'reward': 1.2655498644537573, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 1.27)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: left, reward: 1.55362437031
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 15, 't': 15, 'action': 'left', 'reward': 1.5536243703111001, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.55)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: 2.3150790296
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': 2.315079029597143, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.32)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: right, reward: 0.470798273453
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 0.470798273453237, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.47)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: left, reward: 1.69516058715
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 18, 'action': 'left', 'reward': 1.6951605871528512, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.70)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: -0.226798418418
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': -0.22679841841770398, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent drove forward instead of right. (rewarded -0.23)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: 0.780885058472
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'left'), 'deadline': 10, 't': 20, 'action': 'forward', 'reward': 0.7808850584716596, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'left')
Agent drove forward instead of right. (rewarded 0.78)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: right, reward: 1.72791549534
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 1.727915495338573, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.73)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 1.8560362176
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 1.8560362175960616, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.86)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 1.04781755446
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 1.0478175544647805, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 1.05)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: forward, reward: 1.01895445864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 6, 't': 24, 'action': 'forward', 'reward': 1.0189544586443564, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 1.02)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: forward, reward: 0.469189987076
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 5, 't': 25, 'action': 'forward', 'reward': 0.4691899870755174, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.47)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: right, reward: 0.650385795411
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 26, 'action': 'right', 'reward': 0.6503857954109573, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 0.65)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.77260475504
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 3, 't': 27, 'action': None, 'reward': 1.772604755037684, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.77)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 0.281264471313
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 2, 't': 28, 'action': None, 'reward': 0.2812644713132191, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 0.28)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 0.436857629797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', 'right'), 'deadline': 1, 't': 29, 'action': None, 'reward': 0.4368576297969389, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'right', 'forward', 'right')
Agent properly idled at a red light. (rewarded 0.44)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 115
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (1, 6), deadline = 25
Simulating trial. . .
epsilon = 0.3166; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3166; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: forward, reward: 2.85522800999
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 2.8552280099945415, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.86)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.03519577094
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.0351957709416657, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.04)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: forward, reward: -10.3766183714
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': -10.376618371386396, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.38)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: 2.4535008921
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.4535008920991555, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.45)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: right, reward: 0.353866014339
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 0.3538660143386164, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent drove right instead of forward. (rewarded 0.35)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 2.84375486831
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.8437548683115472, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.84)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 2.88766181721
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.8876618172065536, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.89)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: 1.36520684174
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 1.3652068417430008, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.37)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.22554353753
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.225543537532949, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.23)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.88528735986
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.885287359860427, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.89)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: 2.26466477627
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 2.2646647762694716, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.26)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 2.04040659979
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.040406599792707, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.04)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 0.899387140082
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 13, 't': 12, 'action': None, 'reward': 0.8993871400818378, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 0.90)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: -4.27512575677
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': -4.27512575677406, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.28)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: forward, reward: 1.8832703076
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 1.8832703075982924, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.88)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 1.03040462465
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.0304046246521315, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.03)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 1.20880700011
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 1.2088070001108107, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent drove right instead of forward. (rewarded 1.21)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: -0.221408572976
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': -0.22140857297626249, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove forward instead of left. (rewarded -0.22)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.32364734382
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.3236473438158194, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.32)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.77811142296
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', 'left'), 'deadline': 6, 't': 19, 'action': None, 'reward': 1.778111422957904, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', 'left')
Agent properly idled at a red light. (rewarded 1.78)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 2.0433864457
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 2.0433864456989856, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.04)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: left, reward: 1.12199615177
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 1.1219961517684642, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 1.12)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: -0.33363075788
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': -0.3336307578796731, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove forward instead of left. (rewarded -0.33)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: 0.360979297261
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 2, 't': 23, 'action': 'left', 'reward': 0.3609792972613599, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 0.36)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 0.997142907733
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.9971429077331626, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.00)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 116
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (3, 5), deadline = 25
Simulating trial. . .
epsilon = 0.3135; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.85709252867
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.8570925286729096, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.86)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: left, reward: -39.945420771
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -39.94542077095434, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.95)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: right, reward: 1.42263451682
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.4226345168167684, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 1.42)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 1.81500381517
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.8150038151731782, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.82)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 1.39183431074
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.3918343107433595, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.39)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: 1.46186415417
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.4618641541703448, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 1.46)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.50081849724
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.500818497242412, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.50)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 2.65229545419
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 2.652295454186069, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.65)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: left, reward: -9.0175582091
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': 'left', 'reward': -9.017558209095675, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.02)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: -10.8940716884
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': -10.894071688415348, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.89)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 0.0399793644802
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 0.039979364480211, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.04)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: 1.50781823127
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.5078182312691855, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'left')
Agent drove forward instead of left. (rewarded 1.51)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 1.00483296848
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.0048329684798751, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 1.00)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: -10.8012133081
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': -10.801213308095551, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent attempted driving left through a red light. (rewarded -10.80)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: forward, reward: -9.05895690968
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': -9.058956909683634, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -9.06)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: 2.4447968077
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.444796807695176, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.44)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: 0.698999153726
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 0.698999153725891, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 0.70)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: 0.815773440805
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 0.8157734408045629, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.82)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 2.09051050207
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 2.090510502068661, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.09)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: 1.38456983364
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 1.3845698336427845, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.38)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: forward, reward: 1.13672860099
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 5, 't': 20, 'action': 'forward', 'reward': 1.1367286009949957, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 1.14)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: left, reward: 0.385993546437
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 0.38599354643713946, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 0.39)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: left, reward: -0.705023486411
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', None), 'deadline': 3, 't': 22, 'action': 'left', 'reward': -0.7050234864105753, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', None)
Agent drove left instead of right. (rewarded -0.71)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: forward, reward: 0.621608004004
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 23, 'action': 'forward', 'reward': 0.6216080040036784, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.62)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: forward, reward: -39.5423309104
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': -39.54233091039657, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.54)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 117
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (4, 3), deadline = 20
Simulating trial. . .
epsilon = 0.3104; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: -39.7118898873
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -39.71188988732424, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.71)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: -39.3517893117
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -39.35178931173827, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.35)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.93552364553
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.9355236455277842, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.94)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.15936850446
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.159368504458129, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.16)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.96535205922
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.9653520592193114, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.97)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.14301534421
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.1430153442134374, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.14)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: left, reward: 1.14515485246
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.1451548524619293, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.15)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: -4.86948009259
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 13, 't': 7, 'action': None, 'reward': -4.869480092590571, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.87)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 1.26585732966
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.2658573296596805, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', None)
Agent drove right instead of left. (rewarded 1.27)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 1.22025604729
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.2202560472877995, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.22)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: 2.04715176106
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 2.0471517610607433, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.05)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 2.26133498042
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.261334980419585, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.26)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: -9.38386229679
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -9.383862296794359, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.38)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.05902941128
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.0590294112782472, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.06)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.18955623878
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.1895562387803968, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 1.19)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: 2.1215297109
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.121529710898112, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.12)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 0.209024152204
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.20902415220367, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 0.21)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 0.227353078177
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.2273530781768086, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.23)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 1.38853609202
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.3885360920171668, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.39)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: left, reward: 1.407880029
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 1.407880029002045, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.41)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 118
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (8, 3), deadline = 30
Simulating trial. . .
epsilon = 0.3073; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3073; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3073; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3073; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 1.56006921615
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.5600692161518979, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'left')
Agent drove right instead of left. (rewarded 1.56)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: -5.96580048431
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', 'right', None), 'deadline': 29, 't': 1, 'action': None, 'reward': -5.965800484314544, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.97)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 2.38321833438
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 2.383218334381084, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.38)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: forward, reward: 0.0620296828038
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 0.06202968280375909, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.06)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: 1.07427958147
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 1.0742795814741615, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.07)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: -5.91854805035
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'forward'), 'deadline': 25, 't': 5, 'action': None, 'reward': -5.918548050345083, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.92)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: right, reward: 1.87128309987
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 1.8712830998713246, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 1.87)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 1.15147785375
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.1514778537528487, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.15)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: -9.69495139855
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': -9.694951398545273, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.69)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 1.94831702155
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.9483170215512837, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.95)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 2.32555282386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.3255528238620937, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.33)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 1.40553812177
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 1.405538121769374, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.41)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: -10.7546704297
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 18, 't': 12, 'action': 'left', 'reward': -10.754670429744559, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -10.75)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 0.860032419948
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 0.8600324199484113, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.86)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.70069011483
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.7006901148299314, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.70)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: 0.759973219248
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 0.7599732192477371, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.76)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 1.35478833653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 14, 't': 16, 'action': None, 'reward': 1.3547883365327433, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.35)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 1.65671621693
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': 1.656716216930901, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.66)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 2.21597858286
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 18, 'action': None, 'reward': 2.2159785828584466, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.22)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: left, reward: -9.58460479114
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 11, 't': 19, 'action': 'left', 'reward': -9.584604791144933, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.58)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 0.610645520243
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 10, 't': 20, 'action': None, 'reward': 0.6106455202428085, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 0.61)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: forward, reward: 2.05549887405
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 2.0554988740508726, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.06)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 0.335786551448
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 0.3357865514476803, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent drove right instead of forward. (rewarded 0.34)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: 0.537287290779
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 23, 'action': 'left', 'reward': 0.5372872907792017, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.54)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: -10.923438163
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': 'forward', 'reward': -10.923438163039354, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.92)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 0.98673639784
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 25, 'action': None, 'reward': 0.9867363978397561, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.99)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: left, reward: 1.36550737764
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 4, 't': 26, 'action': 'left', 'reward': 1.3655073776433604, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.37)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: forward, reward: 1.31120906517
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 3, 't': 27, 'action': 'forward', 'reward': 1.3112090651669368, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.31)
7% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 119
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (8, 6), deadline = 20
Simulating trial. . .
epsilon = 0.3042; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 1.43642809551
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.4364280955081374, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.44)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 1.44984594596
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.4498459459641637, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.45)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 1.75582599848
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.755825998479816, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.76)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 1.12261790996
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.1226179099564486, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.12)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 0.953740354347
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 0.9537403543465195, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.95)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 1.20314648048
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.2031464804809913, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.20)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 2.26966132734
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.2696613273426856, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.27)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: 2.55948724636
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 2.559487246359628, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.56)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.74702614183
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.747026141825299, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.75)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.48186729276
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.4818672927599335, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.48)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 2.09161796209
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.091617962091804, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', 'left')
Agent properly idled at a red light. (rewarded 2.09)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: left, reward: 0.843433995111
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 0.8434339951110501, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 0.84)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: forward, reward: -9.7802224403
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': -9.780222440295141, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.78)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: left, reward: 1.01131281104
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.0113128110423064, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 1.01)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.15245324062
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.1524532406236145, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.15)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 0.498824261695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 0.4988242616952345, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.50)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: right, reward: 0.928194041071
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.9281940410713618, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 0.93)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 2.15048672818
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 2.150486728176987, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.15)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: right, reward: 0.250933230359
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 0.25093323035877213, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.25)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 0.410408269049
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.4104082690485996, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 0.41)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 120
\-------------------------
Environment.reset(): Trial set up with start = (5, 5), destination = (2, 3), deadline = 25
Simulating trial. . .
epsilon = 0.3012; alpha = 0.1000
Simulating trial. . .
epsilon = 0.3012; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: left, reward: 1.96366153939
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 1.9636615393904395, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.96)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: left, reward: -40.066147501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'left'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -40.066147501005574, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.07)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 1.30773016656
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.3077301665564993, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.31)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 2.35495146061
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.3549514606128827, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.35)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: 0.314323755918
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 0.3143237559182016, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 0.31)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 2.71026217867
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.7102621786652614, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.71)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.93605285116
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.9360528511615713, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.94)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 2.67601197223
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 2.676011972229039, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 2.68)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 1.49072565633
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.4907256563281317, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 1.49)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: left, reward: -10.9586652474
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': -10.958665247363369, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.96)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 2.76999487334
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.769994873338126, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.77)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.75058707901
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.7505870790143212, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.75)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: 1.13267604971
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 13, 't': 12, 'action': 'left', 'reward': 1.1326760497086314, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.13)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 0.859446845883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 12, 't': 13, 'action': None, 'reward': 0.8594468458832534, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 0.86)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.04050796745
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.0405079674475484, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.04)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.55536988293
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.5553698829328537, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.56)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 2.53490544035
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 2.5349054403543194, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.53)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: -4.7412643898
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': None, 'reward': -4.741264389796459, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.74)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: 2.22162064301
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': 2.221620643010673, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.22)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: left, reward: 1.83219410727
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 1.8321941072680028, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.83)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 1.68707872563
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.687078725625216, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.69)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: 2.03704866599
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 2.037048665986597, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 2.04)
12% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 121
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (3, 6), deadline = 25
Simulating trial. . .
epsilon = 0.2982; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 2.05897622764
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.058976227644628, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.06)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: -9.78421597505
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -9.784215975048022, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.78)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 1.3564600766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.3564600765962944, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.36)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 1.31642565321
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.3164256532138927, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.32)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 1.26558693061
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.265586930612049, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.27)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 1.82328078866
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.823280788659661, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.82)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 0.73253812644
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.7325381264399092, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.73)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.36685113741
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.3668511374075312, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.37)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.37492814057
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.3749281405715277, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.37)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: -9.31264321669
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': -9.312643216686835, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.31)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 2.20900892174
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 2.209008921740228, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.21)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: 1.45502524011
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.455025240112381, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 1.46)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 2.45748801311
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 2.457488013114663, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.46)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 2.7007680965
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 2.7007680964960272, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.70)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: -0.059630850858
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': -0.05963085085804665, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded -0.06)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: 2.56899491987
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 2.5689949198712734, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.57)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 0.736477031761
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 0.7364770317606291, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.74)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 1.38118704861
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 1.3811870486080504, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.38)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 0.227960650547
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 0.227960650546694, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent drove right instead of forward. (rewarded 0.23)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: forward, reward: -9.00670778567
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': -9.00670778567031, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -9.01)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: -0.437752209754
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'right'), 'deadline': 5, 't': 20, 'action': 'right', 'reward': -0.4377522097536817, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'right')
Agent drove right instead of left. (rewarded -0.44)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 1.24413833215
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 1.2441383321549573, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.24)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: forward, reward: -9.84738894534
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': -9.847388945335249, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.85)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 1.49011315425
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 2, 't': 23, 'action': 'right', 'reward': 1.4901131542496917, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.49)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: left, reward: -0.816063016459
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 1, 't': 24, 'action': 'left', 'reward': -0.8160630164587666, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded -0.82)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 122
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (5, 5), deadline = 35
Simulating trial. . .
epsilon = 0.2952; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2952; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 0.346976977695
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'forward'), 'deadline': 35, 't': 0, 'action': 'right', 'reward': 0.3469769776952707, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.35)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 2.04151054793
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 34, 't': 1, 'action': None, 'reward': 2.0415105479338256, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.04)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 2.36884233237
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 33, 't': 2, 'action': None, 'reward': 2.3688423323706456, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.37)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: forward, reward: 1.94545405237
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', None), 'deadline': 32, 't': 3, 'action': 'forward', 'reward': 1.9454540523711057, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', None)
Agent drove forward instead of left. (rewarded 1.95)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 2.67647548865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 31, 't': 4, 'action': None, 'reward': 2.6764754886498934, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.68)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.15885375005
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 30, 't': 5, 'action': None, 'reward': 1.1588537500490232, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.16)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.81131059805
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 29, 't': 6, 'action': None, 'reward': 1.8113105980516435, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.81)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: left, reward: 1.47720671843
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 28, 't': 7, 'action': 'left', 'reward': 1.477206718426557, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.48)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 1.90290822097
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 27, 't': 8, 'action': 'forward', 'reward': 1.9029082209664487, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.90)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: -20.1835817558
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'forward'), 'deadline': 26, 't': 9, 'action': 'left', 'reward': -20.183581755802994, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.18)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 0.00354655517585
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 25, 't': 10, 'action': 'right', 'reward': 0.0035465551758487868, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.00)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: left, reward: 1.91907340466
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 24, 't': 11, 'action': 'left', 'reward': 1.9190734046560531, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.92)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: right, reward: 0.974478781803
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 23, 't': 12, 'action': 'right', 'reward': 0.9744787818029069, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.97)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: left, reward: 2.39413555277
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 22, 't': 13, 'action': 'left', 'reward': 2.394135552770347, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.39)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 2.19076574933
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 21, 't': 14, 'action': None, 'reward': 2.1907657493306174, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.19)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.73120566466
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 20, 't': 15, 'action': None, 'reward': 1.7312056646601033, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.73)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: left, reward: 1.49929913201
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 19, 't': 16, 'action': 'left', 'reward': 1.4992991320065365, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.50)
51% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 123
\-------------------------
Environment.reset(): Trial set up with start = (4, 7), destination = (2, 3), deadline = 20
Simulating trial. . .
epsilon = 0.2923; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2923; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: right, reward: 0.830783064297
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.8307830642966483, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 0.83)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: left, reward: 1.13639704376
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.136397043755831, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent drove left instead of right. (rewarded 1.14)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: forward, reward: -40.1651113374
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': -40.165111337417635, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.17)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 2.58919408271
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.5891940827126207, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 2.59)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 1.9021907759
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.90219077589649, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.90)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: forward, reward: 0.377656543674
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 0.37765654367418233, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.38)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: None, reward: 2.2021072111
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.2021072110973634, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.20)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: left, reward: 2.51129698429
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 2.5112969842909667, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.51)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 2.11838762213
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.118387622125967, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.12)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 2.28505870684
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.2850587068401578, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.29)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: 1.85669890176
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.8566989017622622, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.86)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 0.782079538387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 0.7820795383868151, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.78)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 0.530074899175
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.5300748991745943, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.53)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: forward, reward: 1.15797475067
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.1579747506711136, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove forward instead of left. (rewarded 1.16)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: None, reward: 0.977504489868
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.9775044898676568, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.98)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: None, reward: 2.38975356388
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.3897535638792125, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.39)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: 0.632260513191
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 0.6322605131910837, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.63)
15% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 124
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (5, 7), deadline = 25
Simulating trial. . .
epsilon = 0.2894; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 0.457779245758
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 0.45777924575820594, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.46)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: forward, reward: -9.96791825775
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': -9.96791825775033, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.97)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 0.82691266427
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 0.8269126642699294, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 0.83)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 2.5673835663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.5673835662964177, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.57)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 0.148283395174
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 0.1482833951744823, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.15)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: left, reward: 1.76057341779
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.7605734177881056, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 1.76)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: 2.72956898146
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.729568981458662, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.73)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: 1.9924221551
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.9924221550970338, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.99)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 1.21764855085
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.2176485508548889, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.22)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 1.33559058849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.335590588487674, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.34)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: left, reward: 1.03185441628
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 1.031854416283391, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.03)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 125
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (3, 6), deadline = 30
Simulating trial. . .
epsilon = 0.2865; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2865; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 1.20159250021
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'right'), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.201592500213007, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'right')
Agent properly idled at a red light. (rewarded 1.20)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: forward, reward: -9.80560218734
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': -9.805602187338241, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.81)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 1.00157550648
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.0015755064790641, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.00)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 1.70246785652
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 1.7024678565193354, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 1.70)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: -9.64210001108
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': -9.642100011083063, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -9.64)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 1.61216729352
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.6121672935207312, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.61)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: 1.40649537389
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 1.4064953738927017, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove left instead of right. (rewarded 1.41)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: 2.44288902415
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 2.4428890241521817, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.44)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: 2.57309408471
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': 2.5730940847086474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.57)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: -10.9888138636
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': -10.98881386362866, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.99)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: forward, reward: 2.61624144171
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 2.616241441705903, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 2.62)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: left, reward: 1.33587234107
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 1.3358723410670157, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 1.34)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 2.82797889636
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 2.82797889635678, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.83)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 1.96666073768
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 1.9666607376831053, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.97)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: 1.19730122162
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.1973012216152714, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.20)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: 2.65031215965
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 15, 't': 15, 'action': 'left', 'reward': 2.650312159651275, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.65)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 0.430976989373
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 0.4309769893732842, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.43)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: forward, reward: -0.269285463344
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': -0.26928546334430004, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded -0.27)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 2.06535767054
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', 'forward'), 'deadline': 12, 't': 18, 'action': None, 'reward': 2.065357670540733, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.07)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: -0.00437550141893
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': -0.004375501418934502, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded -0.00)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.09075888731
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 20, 'action': None, 'reward': 1.0907588873052034, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.09)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 0.394862683045
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 0.3948626830448535, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.39)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: -0.445505258358
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 8, 't': 22, 'action': 'left', 'reward': -0.44550525835825594, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded -0.45)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: left, reward: 0.42031233271
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 7, 't': 23, 'action': 'left', 'reward': 0.4203123327104442, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.42)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 0.778929606196
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 0.7789296061962083, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 0.78)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.53668311146
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 5, 't': 25, 'action': None, 'reward': 1.5366831114601247, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.54)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: 0.911979963963
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 26, 'action': 'forward', 'reward': 0.9119799639625188, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.91)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 0.634847392362
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 27, 'action': 'right', 'reward': 0.6348473923618012, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.63)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 1.19118144967
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 2, 't': 28, 'action': None, 'reward': 1.191181449672014, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.19)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 0.245538254271
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'left'), 'deadline': 1, 't': 29, 'action': None, 'reward': 0.2455382542710285, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'forward', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.25)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 126
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (3, 3), deadline = 20
Simulating trial. . .
epsilon = 0.2837; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: None, reward: 0.685286121654
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 0.6852861216535755, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.69)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: left, reward: 1.31731602396
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.3173160239597934, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.32)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: 0.649524944491
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.6495249444907522, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent drove right instead of forward. (rewarded 0.65)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: left, reward: 2.51914694439
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 2.5191469443850165, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.52)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.59439795823
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.594397958233293, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.59)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 1.75132171413
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.7513217141274138, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.75)
70% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 127
\-------------------------
Environment.reset(): Trial set up with start = (2, 6), destination = (5, 3), deadline = 30
Simulating trial. . .
epsilon = 0.2808; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2808; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2808; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: 2.50846288481
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 2.5084628848109096, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.51)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: forward, reward: 2.22440876704
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 2.2244087670381982, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.22)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 2.34857765634
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.3485776563408365, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.35)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: forward, reward: 2.38119417503
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 2.381194175029944, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.38)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 1.94743968512
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 1.947439685117348, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.95)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: left, reward: 1.80874623524
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 1.808746235243587, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.81)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: left, reward: 0.585769860729
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 0.5857698607293834, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove left instead of right. (rewarded 0.59)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: left, reward: 2.25889015098
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 2.258890150978851, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.26)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.63926194272
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.6392619427241497, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.64)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: forward, reward: 1.10716262512
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.1071626251239515, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 1.11)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: None, reward: 1.12776652088
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.1277665208830194, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.13)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: 2.14035824793
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 2.140358247928855, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.14)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 0.436576355532
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 0.4365763555317833, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.44)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.40028934284
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 13, 'action': None, 'reward': 1.4002893428431293, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.40)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 0.790213971976
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 16, 't': 14, 'action': None, 'reward': 0.7902139719764856, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 0.79)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.67313610272
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.6731361027180658, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.67)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: left, reward: 0.78747747353
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 0.7874774735297394, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.79)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: 0.815907458665
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': 0.8159074586654915, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 0.82)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: -0.280182426319
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 12, 't': 18, 'action': 'right', 'reward': -0.2801824263194843, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded -0.28)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 2.19021181313
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 11, 't': 19, 'action': None, 'reward': 2.1902118131329784, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.19)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 0.654718252049
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 20, 'action': None, 'reward': 0.6547182520494339, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.65)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: -4.88644021587
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 9, 't': 21, 'action': None, 'reward': -4.8864402158719855, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.89)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 1.25642970817
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 1.256429708168284, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.26)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: left, reward: -0.508544971324
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 23, 'action': 'left', 'reward': -0.5085449713242932, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded -0.51)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: -4.26064609045
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'right'), 'deadline': 6, 't': 24, 'action': None, 'reward': -4.260646090447579, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.26)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 0.927493508762
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 0.9274935087622211, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.93)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: 1.58226512074
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 26, 'action': 'left', 'reward': 1.582265120737936, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.58)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 2.04537047228
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 3, 't': 27, 'action': 'forward', 'reward': 2.0453704722784662, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.05)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 0.225805824529
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 2, 't': 28, 'action': None, 'reward': 0.22580582452854214, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.23)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: -9.984306657
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 1, 't': 29, 'action': 'left', 'reward': -9.984306657002378, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.98)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 128
\-------------------------
Environment.reset(): Trial set up with start = (5, 7), destination = (1, 3), deadline = 30
Simulating trial. . .
epsilon = 0.2780; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.9790535552
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.9790535551975363, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.98)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.0022869709
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.0022869708983044, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.00)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: right, reward: 0.219319025154
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 0.21931902515369728, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.22)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: -5.57573738764
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 27, 't': 3, 'action': None, 'reward': -5.575737387643214, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.58)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 1.9048869423
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 1.9048869422952508, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 1.90)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 1.13491041706
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.1349104170590478, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 1.13)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: left, reward: 2.2666055262
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 2.2666055261954714, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.27)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 2.71049593654
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 2.710495936537006, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.71)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.24099134024
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.2409913402399155, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.24)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: -10.3017437853
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': -10.301743785278463, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent attempted driving left through a red light. (rewarded -10.30)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 2.14476002269
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', None), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.144760022693469, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', None)
Agent properly idled at a red light. (rewarded 2.14)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: 2.14467239035
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 2.144672390350202, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.14)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 2.4950505999
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 2.4950505999016084, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.50)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: 0.338307613046
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 0.3383076130459304, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.34)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 1.4136466179
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.413646617900414, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.41)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: left, reward: 1.57832885543
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 15, 't': 15, 'action': 'left', 'reward': 1.5783288554254105, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove left instead of right. (rewarded 1.58)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 0.795606633363
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', 'left'), 'deadline': 14, 't': 16, 'action': None, 'reward': 0.7956066333630958, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.80)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 2.36920943046
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 2.3692094304626754, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 2.37)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 1.55268283911
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 1.5526828391133976, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.55)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.69849680172
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 19, 'action': None, 'reward': 1.698496801719517, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.70)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.20860502065
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 20, 'action': None, 'reward': 1.208605020648558, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.21)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 1.12175219638
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 1.121752196380451, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.12)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 0.693863261195
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 0.6938632611951352, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.69)
23% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 129
\-------------------------
Environment.reset(): Trial set up with start = (8, 7), destination = (4, 7), deadline = 20
Simulating trial. . .
epsilon = 0.2753; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 2.46915844865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.469158448650635, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 2.47)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.19076302575
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.1907630257517758, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.19)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.0326958051
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.0326958051001445, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.03)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 2.30605085052
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.306050850524683, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.31)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 2.59426245513
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.5942624551274434, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 2.59)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.54019056387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.5401905638716809, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.54)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: 1.04332645802
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.0433264580233148, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.04)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 2.37181304056
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.3718130405560345, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.37)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 0.318363576283
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.31836357628303935, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.32)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 1.18116179855
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.181161798545327, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.18)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 1.14395851132
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.1439585113166733, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.14)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: 1.41784715551
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 1.4178471555068162, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.42)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.03910782829
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 1.0391078282920987, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.04)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: 1.96900854541
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.969008545407443, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.97)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: 1.75238408079
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.752384080787938, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.75)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: 1.23764480088
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.2376448008755694, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.24)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: left, reward: 0.88104826366
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 0.8810482636599828, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.88)
15% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 130
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (2, 4), deadline = 25
Simulating trial. . .
epsilon = 0.2725; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2725; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 2.96588230413
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.9658823041259064, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.97)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 2.20454718571
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.204547185712876, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.20)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 1.93413421578
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.9341342157797803, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.93)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: 1.29133885386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.2913388538581514, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.29)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: None, reward: 0.952700517025
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 0.9527005170249072, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.95)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: 0.643431433807
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 0.6434314338074704, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.64)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 1.67932640854
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.679326408543219, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.68)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 0.601363416209
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 0.601363416208561, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.60)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 2.42014095872
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 2.420140958724124, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.42)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: -39.5805585355
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': -39.58055853552188, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.58)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: -39.7074009365
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': -39.70740093645848, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.71)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 2.6576158687
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.657615868700767, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.66)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 0.823726101087
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 13, 't': 12, 'action': None, 'reward': 0.8237261010871908, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 0.82)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 0.180265035178
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 0.1802650351779258, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.18)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: left, reward: 1.97122146606
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 1.9712214660629057, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.97)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: 1.72415621891
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 1.7241562189141595, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 1.72)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 1.0352352912
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'forward', None), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 1.035235291199371, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'forward', None)
Agent followed the waypoint right. (rewarded 1.04)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: -0.0433235721244
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'right', 'right'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': -0.043323572124402965, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'right', 'right')
Agent drove right instead of forward. (rewarded -0.04)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 0.0184883908367
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': 0.018488390836720825, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 0.02)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: -0.378426669705
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': -0.3784266697045181, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded -0.38)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 1.08525319812
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.0852531981170332, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.09)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: forward, reward: 0.305662210018
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 0.30566221001825833, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.31)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 0.625895985682
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', None), 'deadline': 3, 't': 22, 'action': None, 'reward': 0.6258959856819857, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 0.63)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 1.98419608392
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'right'), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.9841960839181736, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.98)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 0.903803303094
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.9038033030938273, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 0.90)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 131
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (5, 4), deadline = 25
Simulating trial. . .
epsilon = 0.2698; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 2.27094076048
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.270940760482148, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.27)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 0.113305078325
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.11330507832463521, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove right instead of forward. (rewarded 0.11)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.13372891904
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.1337289190380664, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.13)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 2.48740827836
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.487408278363415, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.49)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: forward, reward: 0.842794107131
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 0.842794107130516, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove forward instead of left. (rewarded 0.84)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: forward, reward: 0.00876844786477
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 0.008768447864772821, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent drove forward instead of left. (rewarded 0.01)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: left, reward: 2.82833206848
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 2.8283320684757305, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.83)
72% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 132
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (2, 3), deadline = 20
Simulating trial. . .
epsilon = 0.2671; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.67737694591
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.6773769459065235, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.68)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: -20.3458144987
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': -20.34581449866079, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.35)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.03994838387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.0399483838672239, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.04)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: right, reward: 0.196607801516
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 0.19660780151581847, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 0.20)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: -10.5798072427
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -10.579807242658276, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent attempted driving left through a red light. (rewarded -10.58)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 1.3172858195
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.317285819495768, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.32)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: -4.40285335995
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'forward'), 'deadline': 14, 't': 6, 'action': None, 'reward': -4.402853359946068, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.40)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: right, reward: 1.33663501775
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.3366350177469948, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 1.34)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: left, reward: 2.33530855953
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 2.335308559526509, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.34)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: forward, reward: 1.22204402811
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.222044028112151, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.22)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 1.79630062209
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.7963006220879727, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.80)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: right, reward: 0.0954461520111
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'right'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.09544615201105167, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'right')
Agent drove right instead of forward. (rewarded 0.10)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 0.914437783746
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.914437783745963, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 0.91)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 1.48461590475
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.4846159047484422, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.48)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 2.32245572699
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.322455726990195, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 2.32)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 0.968727488143
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.9687274881430548, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.97)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: 0.737793303753
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 0.7377933037525806, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.74)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: left, reward: 1.85672319191
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 1.8567231919065035, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.86)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: 0.940021542863
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 0.9400215428631005, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.94)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: left, reward: 2.07516087229
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 2.075160872286559, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.08)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 133
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (7, 2), deadline = 25
Simulating trial. . .
epsilon = 0.2645; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 1.89802191414
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.8980219141419687, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.90)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 0.494793819035
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.49479381903475006, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent drove right instead of forward. (rewarded 0.49)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: -4.37190995248
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 23, 't': 2, 'action': None, 'reward': -4.371909952478492, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.37)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 1.35889436507
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.3588943650667944, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.36)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 2.41261010203
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.4126101020261963, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.41)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 1.94813793478
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.9481379347778354, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.95)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 2.86844288933
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.868442889328458, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.87)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 1.77480115606
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.7748011560578367, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.77)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: forward, reward: -0.038096467314
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -0.038096467313990856, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove forward instead of left. (rewarded -0.04)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: left, reward: 1.53062361015
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.5306236101535935, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.53)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: forward, reward: 1.50331113745
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 1.5033111374512997, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.50)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 134
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (2, 7), deadline = 25
Simulating trial. . .
epsilon = 0.2618; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 1.38459704215
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.3845970421472262, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.38)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 1.00098725333
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.000987253333579, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.00)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 2.05027756308
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 2.0502775630828305, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.05)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 1.49367635751
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.4936763575143974, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.49)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 2.32748477022
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.327484770223389, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.33)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 2.73619672459
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.736196724593846, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.74)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 0.753230552935
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.7532305529352658, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 0.75)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: forward, reward: 0.647994642616
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 0.6479946426163636, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'left')
Agent drove forward instead of right. (rewarded 0.65)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: left, reward: 0.00576248212888
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 0.005762482128879198, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 0.01)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: -39.0696613683
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': -39.06966136834625, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.07)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: left, reward: 0.627461240415
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 0.6274612404149337, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent drove left instead of right. (rewarded 0.63)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 2.4099318989
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 2.409931898895492, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 2.41)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 1.26504186956
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.265041869560827, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.27)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 0.775177105053
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 0.7751771050526743, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.78)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.1665597346
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.166559734603365, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.17)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.56098033367
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.5609803336720314, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.56)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: left, reward: 0.433040325005
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 0.4330403250045628, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 0.43)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.07944382389
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 8, 't': 17, 'action': None, 'reward': 1.0794438238879316, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.08)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 0.739793210587
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 0.7397932105870635, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.74)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 2.23385740819
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 19, 'action': None, 'reward': 2.233857408188827, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.23)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.90688500662
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.906885006621921, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.91)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: forward, reward: 1.76633166627
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 1.7663316662710882, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.77)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: left, reward: -9.77908407866
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 3, 't': 22, 'action': 'left', 'reward': -9.779084078663447, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.78)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 1.15781286391
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.1578128639142151, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.16)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 1.90367121209
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.9036712120943797, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.90)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 135
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (4, 7), deadline = 20
Simulating trial. . .
epsilon = 0.2592; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 2.46899172797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.46899172796768, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.47)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: -9.74041137462
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -9.740411374620635, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.74)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 1.67511678774
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.6751167877432245, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 1.68)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 2.17101284308
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.1710128430830364, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.17)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: left, reward: -19.2801128567
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, 'right', 'forward'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -19.280112856718954, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.28)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: left, reward: 2.32881405975
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.3288140597525606, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.33)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 0.907395796463
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'right'), 'deadline': 14, 't': 6, 'action': None, 'reward': 0.907395796463236, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'right')
Agent properly idled at a red light. (rewarded 0.91)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: left, reward: -40.9795106223
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', 'right'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': -40.97951062225881, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.98)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: -9.74263535599
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -9.742635355993306, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.74)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 2.7872264329
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 2.7872264329024934, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.79)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: -19.647473891
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', 'left'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': -19.64747389102981, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.65)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 1.8140111384
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.814011138397851, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.81)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 1.56779373634
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.5677937363404337, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', 'left')
Agent properly idled at a red light. (rewarded 1.57)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 1.5077226824
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.5077226823969747, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.51)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: 0.959250940053
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.9592509400525411, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.96)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: left, reward: 1.25972153665
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 1.259721536653029, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.26)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: left, reward: 0.854482725888
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 0.8544827258875434, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent followed the waypoint left. (rewarded 0.85)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: right, reward: 0.70847791011
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.7084779101095214, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.71)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 1.9025313136
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.902531313598473, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.90)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 0.40929801768
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.40929801767995544, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.41)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 136
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (3, 2), deadline = 20
Simulating trial. . .
epsilon = 0.2567; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 1.26542830775
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'right'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.2654283077546267, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'right')
Agent drove right instead of left. (rewarded 1.27)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 2.78617504339
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.7861750433871464, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.79)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 1.42510477105
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.4251047710533467, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.43)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 1.19708739324
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.197087393239483, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 1.20)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.82807415443
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.8280741544287715, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.83)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 2.64991658623
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.649916586225253, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.65)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.09655070889
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.0965507088938165, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.10)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: left, reward: 0.872030538232
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 0.8720305382320261, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 0.87)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: left, reward: 1.6701751722
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.6701751722007494, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.67)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: left, reward: -0.146761760377
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': -0.14676176037708588, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded -0.15)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: -5.31184135545
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': -5.311841355454067, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.31)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: 1.76930065345
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.7693006534508358, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.77)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 0.793050887671
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.7930508876710056, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.79)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.42101319771
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.4210131977070997, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.42)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: 1.46350516147
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.4635051614711327, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.46)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: 0.862846534964
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.8628465349640804, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.86)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: 1.96241235738
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.9624123573824874, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.96)
15% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 137
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (8, 6), deadline = 30
Simulating trial. . .
epsilon = 0.2541; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2541; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2541; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: left, reward: 2.95128230733
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 2.951282307334366, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 2.95)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: 1.14466909885
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 1.1446690988549961, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.14)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 1.9703798829
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.9703798828970642, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.97)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: forward, reward: 0.535576726872
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 0.5355767268724002, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.54)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: 1.57728178046
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 1.5772817804573962, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.58)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 2.34570760771
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.345707607707805, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.35)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 1.95225237641
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.9522523764115014, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.95)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 1.70520362171
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.7052036217123752, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.71)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: -9.41321344421
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': -9.413213444212808, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.41)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 1.76978803265
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 1.769788032653911, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.77)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: right, reward: 1.34312290332
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.34312290332324, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.34)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 0.618234896103
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 0.6182348961033662, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.62)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 1.38939635482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.3893963548179193, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.39)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: left, reward: 2.53574812111
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 2.5357481211108714, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.54)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 2.62552784723
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 2.625527847228999, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.63)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 2.17765919803
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': None, 'reward': 2.1776591980309044, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.18)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: left, reward: 1.97205785607
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 1.9720578560664102, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.97)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 1.19812467132
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': 1.1981246713243927, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.20)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 2.2277606591
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 12, 't': 18, 'action': None, 'reward': 2.2277606590974655, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.23)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 2.35997967242
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'right'), 'deadline': 11, 't': 19, 'action': None, 'reward': 2.359979672422531, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'right')
Agent properly idled at a red light. (rewarded 2.36)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 1.88125634528
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 20, 'action': None, 'reward': 1.8812563452788342, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.88)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: forward, reward: 1.55958022951
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 1.5595802295057137, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.56)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: 2.49002626993
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': 2.4900262699303024, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.49)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 0.492675222023
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 7, 't': 23, 'action': 'forward', 'reward': 0.492675222022639, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent drove forward instead of right. (rewarded 0.49)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 1.52971597429
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': None, 'reward': 1.5297159742851514, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.53)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: -4.88056838758
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 25, 'action': None, 'reward': -4.880568387583462, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.88)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 2.22997788045
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 4, 't': 26, 'action': 'right', 'reward': 2.229977880452676, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.23)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 1.26677866446
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 3, 't': 27, 'action': 'right', 'reward': 1.266778664462565, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.27)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: 0.917799489891
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 2, 't': 28, 'action': 'left', 'reward': 0.9177994898914463, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 0.92)
3% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 138
\-------------------------
Environment.reset(): Trial set up with start = (3, 2), destination = (5, 4), deadline = 20
Simulating trial. . .
epsilon = 0.2516; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 1.43973556722
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', 'right'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.439735567224758, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', 'right')
Agent drove forward instead of left. (rewarded 1.44)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: left, reward: 2.90011220339
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 2.900112203388936, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.90)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: left, reward: 1.73440802109
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.7344080210859145, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 1.73)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 1.16467657313
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.164676573129659, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 1.16)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 1.56802651152
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.5680265115173544, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 1.57)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: forward, reward: 1.44267453043
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.4426745304342437, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded 1.44)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 2.28552358384
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.2855235838419996, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.29)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: left, reward: 1.1800867695
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.1800867694978365, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.18)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: 1.63005052909
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.6300505290902434, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 1.63)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: -4.72893814313
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'forward'), 'deadline': 11, 't': 9, 'action': None, 'reward': -4.728938143133156, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.73)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: forward, reward: 0.169586555596
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 0.16958655559596403, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 0.17)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: forward, reward: 0.0794040771913
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.07940407719131604, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', None)
Agent drove forward instead of left. (rewarded 0.08)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 0.253507253651
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.2535072536508869, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 0.25)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 2.52650464862
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 2.526504648622302, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 2.53)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 1.55056636062
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.5505663606190576, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.55)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: left, reward: 0.0643958938822
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 0.06439589388222133, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', None)
Agent drove left instead of forward. (rewarded 0.06)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 0.995630344521
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.9956303445205499, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.00)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 1.21690092169
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 1.2169009216871713, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.22)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.55635359693
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.5563535969269304, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', 'left')
Agent properly idled at a red light. (rewarded 1.56)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: 0.39841414203
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 0.3984141420296543, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.40)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 139
\-------------------------
Environment.reset(): Trial set up with start = (3, 4), destination = (6, 5), deadline = 20
Simulating trial. . .
epsilon = 0.2491; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: 1.30455414148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'right'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.3045541414756097, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'right')
Agent properly idled at a red light. (rewarded 1.30)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 0.162673082145
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.16267308214457132, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 0.16)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 0.755686116831
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.755686116831057, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.76)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: left, reward: 2.49298733874
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 2.492987338738457, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.49)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.86914073285
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.8691407328531704, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.87)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.5242624808
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.524262480803171, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.52)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 1.56233530833
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.5623353083252285, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent drove right instead of forward. (rewarded 1.56)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 1.34893409182
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.348934091817056, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'forward')
Agent drove right instead of left. (rewarded 1.35)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: -40.758063622
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -40.75806362200305, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.76)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.64251816083
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.6425181608305204, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.64)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.30765480571
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.3076548057136737, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.31)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: 1.0159592295
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 1.01595922950445, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.02)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 1.71262003416
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.7126200341639704, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.71)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 1.91080600191
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.9108060019077129, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.91)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: -10.3543389995
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': -10.354338999470123, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.35)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.35319289099
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.3531928909883613, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.35)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: -9.58094339599
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': -9.580943395992028, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.58)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: forward, reward: 1.17536627378
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 1.175366273781227, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.18)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 0.535915934203
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.5359159342032334, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.54)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: 1.28484504574
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 1.2848450457396676, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.28)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 140
\-------------------------
Environment.reset(): Trial set up with start = (4, 7), destination = (7, 3), deadline = 25
Simulating trial. . .
epsilon = 0.2466; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: forward, reward: -40.1184503686
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'left'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -40.118450368634825, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.12)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: right, reward: 1.18963648088
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.1896364808797308, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.19)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: 1.59750137994
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 1.597501379943621, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.60)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 2.51738040277
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.517380402767216, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.52)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.78214757217
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.782147572174121, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.78)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: 1.07448029765
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.0744802976468901, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.07)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 1.49715887161
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.497158871610944, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.50)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: 0.112724005662
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 0.11272400566233531, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.11)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 0.900171504294
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 0.9001715042935229, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove forward instead of right. (rewarded 0.90)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 2.65855933633
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 2.658559336327777, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 2.66)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 2.63488048825
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 2.634880488249034, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.63)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 1.34281236382
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.3428123638228178, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.34)
52% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 141
\-------------------------
Environment.reset(): Trial set up with start = (2, 7), destination = (7, 6), deadline = 20
Simulating trial. . .
epsilon = 0.2441; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: forward, reward: 0.88603333422
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 0.8860333342203277, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.89)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 1.62916020093
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.6291602009278596, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.63)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 2.94648156111
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.9464815611052924, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.95)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: forward, reward: -10.9336888394
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': -10.933688839411202, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent attempted driving forward through a red light. (rewarded -10.93)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 2.77147424179
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.771474241794915, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.77)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 1.91146585413
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.9114658541254228, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.91)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: 2.82775152573
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.827751525728868, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.83)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.81790427639
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.8179042763874844, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.82)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.38830362488
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.3883036248820118, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.39)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 2.23682718419
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.2368271841913128, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.24)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: left, reward: 1.05653024233
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.0565302423283662, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent drove left instead of forward. (rewarded 1.06)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 1.97960825819
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.9796082581870065, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.98)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 2.42320192447
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 2.423201924469014, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 2.42)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 0.947231677333
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.9472316773326809, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 0.95)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 1.45551181386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.4555118138643848, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', None)
Agent properly idled at a red light. (rewarded 1.46)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: 1.51733153683
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 1.5173315368278242, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.52)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 142
\-------------------------
Environment.reset(): Trial set up with start = (2, 6), destination = (5, 4), deadline = 25
Simulating trial. . .
epsilon = 0.2417; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: right, reward: 1.76110201338
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.7611020133831203, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.76)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: None, reward: 2.4175285366
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.4175285366002166, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 2.42)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 1.18979718996
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.1897971899618405, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.19)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 0.414250638088
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 0.41425063808821605, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 0.41)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: -20.2059527121
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': -20.205952712108317, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.21)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: left, reward: -10.2109625558
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'left', 'reward': -10.21096255578549, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.21)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: forward, reward: 1.52310538941
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.5231053894096371, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.52)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: 2.6377435672
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.637743567199542, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.64)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 1.04097119246
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.0409711924602254, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.04)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 0.497566264266
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 0.4975662642656402, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.50)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 0.891887212047
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 0.8918872120466315, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent followed the waypoint forward. (rewarded 0.89)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 2.45515509859
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'forward'), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.455155098588813, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.46)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.52912032144
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.5291203214412936, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.53)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.74285653158
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.7428565315788822, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.74)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.90663332158
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.9066333215791968, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.91)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 2.15796561864
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.1579656186378875, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.16)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 0.0536955960496
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', 'forward'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 0.053695596049633365, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.05)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: 1.2399374905
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 1.239937490502671, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.24)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.23555272766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.235552727655732, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.24)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 2.36688856886
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 6, 't': 19, 'action': None, 'reward': 2.3668885688645105, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.37)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 2.38890635175
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 2.3889063517507263, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.39)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 0.532981301884
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 0.5329813018838525, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.53)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 0.952561255306
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.9525612553060232, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.95)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: forward, reward: -10.7860239971
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 2, 't': 23, 'action': 'forward', 'reward': -10.786023997061935, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent attempted driving forward through a red light. (rewarded -10.79)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 2.05375645694
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 2.0537564569353064, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.05)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 143
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (6, 6), deadline = 30
Simulating trial. . .
epsilon = 0.2393; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2393; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2393; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2393; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 2.15002425928
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 2.1500242592795633, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.15)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.74340201007
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.7434020100749317, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.74)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.68098995541
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.6809899554114984, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.68)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 2.73540889037
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.7354088903739457, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.74)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.7930829041
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.793082904101328, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.79)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: 1.96065361767
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.9606536176695184, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.96)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 1.07176708271
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 1.0717670827055457, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.07)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: left, reward: -19.1906325801
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'right'), 'deadline': 23, 't': 7, 'action': 'left', 'reward': -19.190632580057567, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.19)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: -0.0680277968841
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', 'forward'), 'deadline': 22, 't': 8, 'action': 'right', 'reward': -0.06802779688412819, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', 'forward')
Agent drove right instead of forward. (rewarded -0.07)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: left, reward: 1.37655465563
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 1.3765546556273618, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.38)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 0.641247203815
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 0.6412472038153911, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.64)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: forward, reward: -9.42047983397
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': -9.420479833968402, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.42)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 0.986960688518
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 0.9869606885175848, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.99)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: left, reward: 1.26148325251
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 1.261483252506038, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.26)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 1.75055167196
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.7505516719627654, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 1.75)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 0.829608482584
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 0.8296084825843801, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.83)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 2.08444315047
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 14, 't': 16, 'action': None, 'reward': 2.0844431504655185, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.08)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: forward, reward: -10.7122548198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': -10.7122548197578, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -10.71)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 1.41948263818
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.4194826381798393, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.42)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 1.6000891178
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 11, 't': 19, 'action': None, 'reward': 1.6000891178036891, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.60)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 2.26549545151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 10, 't': 20, 'action': None, 'reward': 2.2654954515120114, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.27)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: forward, reward: 1.21020452951
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 1.2102045295077708, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.21)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: forward, reward: 0.552738138037
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': 0.5527381380373908, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 0.55)
23% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 144
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (1, 2), deadline = 30
Simulating trial. . .
epsilon = 0.2369; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 1.20004845594
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.2000484559358209, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.20)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 1.39213353215
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.3921335321537156, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.39)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: 2.14034033883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 2.1403403388316167, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 2.14)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: 2.34574063919
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 2.3457406391920514, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.35)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.69627139839
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.6962713983865587, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.70)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: -5.02849783243
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': -5.028497832431656, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.03)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: left, reward: 0.296007818412
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 0.2960078184122975, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.30)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 2.07309259283
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'right'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 2.0730925928289454, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'right')
Agent followed the waypoint right. (rewarded 2.07)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 1.74294947429
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 1.7429494742934097, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.74)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 1.07353052694
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 1.0735305269427131, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.07)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 2.71666358111
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.716663581106113, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.72)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.68845814417
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.6884581441731494, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.69)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.17228655803
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.1722865580260782, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.17)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 2.16236566068
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 2.1623656606794754, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.16)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 1.03673393385
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'right', 'left'), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.0367339338468167, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'right', 'left')
Agent followed the waypoint right. (rewarded 1.04)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.89407694633
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'right'), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.8940769463349738, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'right')
Agent properly idled at a red light. (rewarded 1.89)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 0.80730680958
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 0.80730680958035, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 0.81)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 1.10822636094
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 1.1082263609385135, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.11)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 2.53603363231
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 12, 't': 18, 'action': None, 'reward': 2.5360336323098167, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.54)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: -9.61717884347
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': -9.617178843471695, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.62)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.90385180845
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 20, 'action': None, 'reward': 1.9038518084496863, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.90)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 0.586633673715
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 0.5866336737153941, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.59)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 2.48432412399
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 2.484324123993896, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.48)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: left, reward: 1.82751219337
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 7, 't': 23, 'action': 'left', 'reward': 1.8275121933713505, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.83)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 1.53794779158
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': None, 'reward': 1.5379477915813269, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.54)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: forward, reward: 0.43299195646
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 5, 't': 25, 'action': 'forward', 'reward': 0.43299195645963007, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 0.43)
13% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 145
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (7, 5), deadline = 30
Simulating trial. . .
epsilon = 0.2346; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: 2.7644638013
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': 2.764463801302991, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent followed the waypoint forward. (rewarded 2.76)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 2.20173403837
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.201734038367513, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.20)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 2.18392391005
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.183923910051986, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.18)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 1.98255976226
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.982559762255272, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.98)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 2.34244370264
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 2.3424437026369125, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.34)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: left, reward: 1.32805021023
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 1.3280502102339362, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove left instead of forward. (rewarded 1.33)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: None, reward: 2.10993884998
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.109938849976425, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.11)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 1.85557248416
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.8555724841641263, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 1.86)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: left, reward: 2.48489104628
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': 2.4848910462796203, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.48)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 1.06983365114
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.069833651141032, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.07)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: left, reward: -9.70278689322
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': 'left', 'reward': -9.702786893218605, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.70)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: left, reward: -9.00741475521
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': -9.00741475521082, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -9.01)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 2.35510144305
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.355101443053725, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.36)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: forward, reward: 1.87725153083
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.8772515308309619, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.88)
53% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 146
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (3, 7), deadline = 20
Simulating trial. . .
epsilon = 0.2322; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 0.138596375616
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.13859637561550775, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', 'forward')
Agent drove right instead of left. (rewarded 0.14)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 2.33674609367
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.336746093670496, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.34)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: forward, reward: 0.0628183741465
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'right'), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 0.06281837414645974, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'right')
Agent drove forward instead of right. (rewarded 0.06)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: forward, reward: 1.71916219238
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.7191621923751956, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 1.72)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 1.79451379047
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.7945137904665873, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.79)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 2.8563012564
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.856301256402433, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.86)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.78123883714
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.7812388371400505, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.78)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: -40.9797880987
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -40.97978809868735, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.98)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.55682784542
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.5568278454151616, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.56)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.45963751266
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.4596375126605394, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.46)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 1.14832043624
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.1483204362384893, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.15)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: 1.26677405702
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.2667740570193502, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', None)
Agent drove forward instead of left. (rewarded 1.27)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: -20.548092215
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': -20.548092215028653, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.55)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.31765053735
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.3176505373476892, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.32)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 0.789639815134
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.7896398151337531, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.79)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.63397494087
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.6339749408725466, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.63)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: right, reward: 1.13020883646
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.130208836462022, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.13)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 0.707954842705
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.7079548427046363, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 0.71)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: forward, reward: -0.421880032901
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': -0.4218800329012695, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded -0.42)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.72052273428
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 1.7205227342819651, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.72)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 147
\-------------------------
Environment.reset(): Trial set up with start = (7, 6), destination = (1, 4), deadline = 20
Simulating trial. . .
epsilon = 0.2299; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 2.64911964644
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.6491196464412314, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.65)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: -10.0199039154
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -10.019903915409506, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.02)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 1.43726181554
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.437261815537804, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.44)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 1.24557794491
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.24557794490932, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.25)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: left, reward: 1.84457531661
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.844575316610113, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.84)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.50861982186
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.5086198218553843, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.51)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 2.83632907966
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.836329079660012, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.84)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: -4.8175687937
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': -4.817568793696537, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.82)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 0.826354489149
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.8263544891485096, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent drove right instead of forward. (rewarded 0.83)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: left, reward: 2.78268888639
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 2.7826888863948094, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.78)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 1.6509955168
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.650995516800462, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.65)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: left, reward: 0.954379515025
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 0.9543795150248693, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.95)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 148
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (2, 7), deadline = 25
Simulating trial. . .
epsilon = 0.2276; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2276; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2276; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2276; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2276; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: 1.55328230598
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.5532823059769254, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.55)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 2.59094617429
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.59094617429082, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.59)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.09084402014
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.090844020141937, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.09)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: 2.29706812955
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.297068129554427, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.30)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.32441950698
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.3244195069802291, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.32)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: right, reward: 0.300475158507
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.3004751585071245, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.30)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 0.97444812204
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 19, 't': 6, 'action': None, 'reward': 0.9744481220395829, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 0.97)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: 1.45199346201
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 1.4519934620100408, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.45)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 1.85582197633
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 1.855821976332449, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', None)
Agent followed the waypoint forward. (rewarded 1.86)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: 0.931759152902
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 0.9317591529015945, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 0.93)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: 0.841970998534
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 0.8419709985340433, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent followed the waypoint forward. (rewarded 0.84)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 149
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (8, 3), deadline = 20
Simulating trial. . .
epsilon = 0.2254; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2254; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: left, reward: -9.6057820797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -9.605782079698113, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'left')
Agent attempted driving left through a red light. (rewarded -9.61)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: None, reward: 1.86056707371
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.8605670737121902, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.86)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 0.841632327366
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.841632327365527, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 0.84)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 0.655263273502
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 0.6552632735021645, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 0.66)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 1.45857744615
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.4585774461546896, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.46)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 1.40475257226
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.4047525722571461, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.40)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: right, reward: 2.34799297195
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.3479929719546915, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 2.35)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.73755259432
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.73755259432459, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', 'left')
Agent properly idled at a red light. (rewarded 1.74)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: -10.3600797708
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -10.360079770783443, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent attempted driving left through a red light. (rewarded -10.36)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: -9.73392634069
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': -9.733926340685887, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.73)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: -10.7651072526
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -10.765107252648889, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.77)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: left, reward: 0.113168662336
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 0.11316866233606926, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.11)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: 1.57259846448
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 1.5725984644842943, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 1.57)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: forward, reward: 1.46216659798
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.462166597983002, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent drove forward instead of right. (rewarded 1.46)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: left, reward: -10.2596375419
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -10.25963754185422, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.26)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: left, reward: -9.60881810272
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -9.608818102715112, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.61)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: 0.444102570044
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 0.4441025700441714, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.44)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: 0.935972729228
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.9359727292280176, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent followed the waypoint forward. (rewarded 0.94)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: left, reward: -9.49233012877
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', 'forward'), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -9.492330128768502, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', 'forward')
Agent attempted driving left through a red light. (rewarded -9.49)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: -39.2117782121
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -39.21177821212532, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.21)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 150
\-------------------------
Environment.reset(): Trial set up with start = (7, 2), destination = (3, 6), deadline = 30
Simulating trial. . .
epsilon = 0.2231; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2231; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2231; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: 2.17118004157
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'left'), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 2.1711800415672293, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'left')
Agent followed the waypoint left. (rewarded 2.17)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.8736599992
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.8736599991960925, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.87)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.84764408392
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.8476440839205617, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.85)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.44735729588
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.4473572958821035, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.45)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 1.01026494667
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 1.0102649466684903, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.01)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.77665603724
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.7766560372362319, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.78)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 2.80087870902
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.8008787090150173, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.80)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 2.70192344578
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 2.7019234457778145, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.70)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: -5.12355413256
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'right'), 'deadline': 22, 't': 8, 'action': None, 'reward': -5.123554132557818, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.12)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.54801111399
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.5480111139940775, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.55)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.43874039347
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.4387403934710903, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.44)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 2.25645585163
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.2564558516343833, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.26)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.29957439884
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.2995743988352286, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.30)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.28462522754
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.2846252275383936, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.28)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.03791054951
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.0379105495082095, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.04)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: 1.16877925782
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 15, 'action': 'left', 'reward': 1.1687792578229523, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.17)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: 0.965907089368
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 0.9659070893676733, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.97)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 2.38308320907
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 2.383083209070678, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.38)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: forward, reward: 0.575321944424
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'right'), 'deadline': 12, 't': 18, 'action': 'forward', 'reward': 0.5753219444244437, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'right')
Agent drove forward instead of right. (rewarded 0.58)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: None, reward: 1.14200687562
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 19, 'action': None, 'reward': 1.14200687561996, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.14)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 1.3900978826
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 1.3900978826043557, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.39)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 0.922372812205
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 0.9223728122051131, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.92)
27% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 151
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (4, 3), deadline = 25
Simulating trial. . .
epsilon = 0.2209; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 2.54962615582
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.549626155817066, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.55)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: left, reward: -9.05628121753
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'forward'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -9.056281217527978, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -9.06)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 2.21867963103
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.2186796310300103, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.22)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 2.27396938053
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.2739693805276335, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 2.27)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 1.68826159128
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.6882615912816237, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.69)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 1.59764881433
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.597648814328826, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.60)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: forward, reward: 0.718020336432
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 0.7180203364324714, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.72)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: left, reward: 2.31569924564
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 2.315699245635276, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.32)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: left, reward: 1.86505650995
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 1.8650565099508762, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.87)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: 2.78458770251
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.784587702506367, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.78)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 2.29850813042
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 2.2985081304182855, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.30)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: -5.30115321782
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 14, 't': 11, 'action': None, 'reward': -5.301153217820672, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.30)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 1.41948667425
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'right'), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 1.4194866742481456, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'right')
Agent followed the waypoint forward. (rewarded 1.42)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 1.21475512825
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.2147551282540443, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 1.21)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: forward, reward: 2.12187164764
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 2.1218716476410986, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.12)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 152
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (5, 6), deadline = 30
Simulating trial. . .
epsilon = 0.2187; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: -10.2924910321
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 30, 't': 0, 'action': 'left', 'reward': -10.292491032093421, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.29)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 1.99412688886
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.9941268888615573, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.99)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: -9.67714050563
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': 'left', 'reward': -9.677140505628206, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.68)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: -20.0540068731
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': -20.054006873058245, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.05)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 2.89463810671
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.8946381067094835, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 2.89)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: -9.41651120767
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': -9.416511207668886, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.42)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: forward, reward: 1.23958801964
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 1.2395880196393816, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 1.24)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 0.92723723878
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 23, 't': 7, 'action': None, 'reward': 0.9272372387795582, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 0.93)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 1.25711316335
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.2571131633526176, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 1.26)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: forward, reward: -39.3127292586
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': -39.31272925858792, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.31)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: 1.72871169429
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'forward'), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 1.7287116942888106, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'forward')
Agent drove forward instead of left. (rewarded 1.73)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: 1.54757632197
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 1.5475763219718448, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.55)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.78666366161
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.7866636616125453, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.79)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 0.916282347836
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 0.9162823478356161, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.92)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: left, reward: 1.09123279822
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 16, 't': 14, 'action': 'left', 'reward': 1.0912327982178536, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 1.09)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 2.39933286047
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 2.399332860468484, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.40)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: -4.94327286651
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 14, 't': 16, 'action': None, 'reward': -4.943272866509703, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.94)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: left, reward: 0.155042554785
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 13, 't': 17, 'action': 'left', 'reward': 0.15504255478474782, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.16)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: -4.72332003848
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 12, 't': 18, 'action': None, 'reward': -4.723320038475734, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.72)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: left, reward: -39.0482508183
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 11, 't': 19, 'action': 'left', 'reward': -39.048250818307615, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.05)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: forward, reward: -9.59477917833
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 20, 'action': 'forward', 'reward': -9.59477917832896, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.59)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 1.33755220399
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 1.3375522039949517, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.34)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.06164352881
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 8, 't': 22, 'action': None, 'reward': 1.0616435288083574, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.06)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.17463734067
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 7, 't': 23, 'action': None, 'reward': 1.1746373406730222, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 1.17)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 0.789733734536
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 6, 't': 24, 'action': None, 'reward': 0.7897337345356161, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 0.79)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: left, reward: 0.961828117579
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 5, 't': 25, 'action': 'left', 'reward': 0.9618281175794989, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 0.96)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: left, reward: -10.1674411541
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 4, 't': 26, 'action': 'left', 'reward': -10.16744115412894, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.17)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: forward, reward: 0.246897347654
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'forward'), 'deadline': 3, 't': 27, 'action': 'forward', 'reward': 0.24689734765412763, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'forward')
Agent followed the waypoint forward. (rewarded 0.25)
7% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 153
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (4, 4), deadline = 25
Simulating trial. . .
epsilon = 0.2165; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2165; alpha = 0.1000
Simulating trial. . .
epsilon = 0.2165; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: right, reward: 1.69201319568
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.6920131956781757, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 1.69)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 2.56209434481
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.5620943448088194, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.56)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: -10.6747900034
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': -10.674790003419952, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.67)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 2.10426918718
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.104269187175011, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.10)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 2.26898209577
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.268982095774998, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.27)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 2.71111415813
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.7111141581286278, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.71)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 2.45433646005
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.4543364600530566, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.45)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: -9.36364500482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': -9.363645004815083, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.36)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 2.40815033072
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.4081503307247036, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.41)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 2.49890558773
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.498905587726266, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.50)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 0.829379619153
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.8293796191534474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 0.83)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: 2.13062055171
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 2.130620551711332, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.13)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.3240739341
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.32407393410114, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.32)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 2.4011711972
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 2.401171197202211, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.40)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.04840946087
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.0484094608729946, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.05)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: 1.06132764753
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.0613276475345856, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.06)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: left, reward: 1.70869239966
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 1.7086923996623604, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.71)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 1.86621381487
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 8, 't': 17, 'action': None, 'reward': 1.8662138148698133, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.87)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 1.02165339179
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'right'), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.0216533917860289, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'right')
Agent properly idled at a red light. (rewarded 1.02)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: left, reward: 1.52229850212
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 1.522298502115763, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.52)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 0.937103480001
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 0.9371034800008382, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.94)
16% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 154
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (6, 6), deadline = 20
Simulating trial. . .
epsilon = 0.2144; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: forward, reward: 0.945256486255
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'forward'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 0.9452564862549613, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'forward')
Agent drove forward instead of right. (rewarded 0.95)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 2.0838607233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.083860723298838, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.08)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 2.25518576418
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.2551857641766286, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.26)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 1.10676607598
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.1067660759827358, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.11)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 2.81192882404
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.811928824042419, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.81)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 0.68584210863
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.6858421086297832, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent drove right instead of forward. (rewarded 0.69)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: 1.79515695771
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.7951569577086701, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.80)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 1.24990443858
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.2499044385837643, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.25)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: None, reward: 1.62453246561
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.6245324656080475, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.62)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: forward, reward: 2.09230176077
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 2.0923017607666643, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.09)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 155
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (2, 3), deadline = 20
Simulating trial. . .
epsilon = 0.2122; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 1.74010810933
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.740108109326204, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent drove right instead of left. (rewarded 1.74)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 0.21794666099
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', 'left'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 0.21794666098984294, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', 'left')
Agent drove forward instead of left. (rewarded 0.22)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: 2.91895550284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'left'), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 2.918955502843487, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'left')
Agent followed the waypoint left. (rewarded 2.92)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: 0.926429411526
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 0.9264294115263209, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.93)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: right, reward: -0.0429531722519
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': -0.04295317225194839, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded -0.04)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: -5.28306132616
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': -5.283061326164618, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.28)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: -10.7533967676
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -10.75339676755053, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.75)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.4135683153
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.4135683153019156, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.41)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: -10.2944292415
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -10.29442924149479, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.29)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 0.953563828233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.9535638282332786, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.95)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 1.09119045292
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.0911904529248888, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.09)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.01057356743
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.0105735674288063, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.01)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 0.991991256087
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.9919912560869426, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.99)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.63908379959
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.6390837995866396, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.64)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 1.39144276893
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.3914427689318858, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.39)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.37835755763
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.378357557634643, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.38)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.74908214014
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.7490821401420737, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.75)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: right, reward: 1.00125554769
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 1.0012555476919642, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', None)
Agent drove right instead of left. (rewarded 1.00)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.67731053675
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.677310536746253, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.68)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.25811040488
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.2581104048810017, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.26)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 156
\-------------------------
Environment.reset(): Trial set up with start = (7, 2), destination = (5, 6), deadline = 20
Simulating trial. . .
epsilon = 0.2101; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 0.817780852201
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 0.8177808522010694, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.82)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 0.360125662702
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.3601256627022117, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.36)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 1.61655562503
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.6165556250337412, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.62)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 1.44010662278
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.440106622782631, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.44)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 2.85579933575
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.8557993357519753, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.86)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 1.55108851986
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.5510885198581057, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.55)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 1.81979318604
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.819793186039144, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 1.82)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 1.2538129132
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.2538129132048261, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.25)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 0.278023285396
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.27802328539600396, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.28)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: forward, reward: 0.573794403801
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'forward'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.5737944038005725, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'forward')
Agent drove forward instead of left. (rewarded 0.57)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 1.03300474464
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.033004744641012, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.03)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 1.45573035354
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.4557303535443495, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.46)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 2.04125554952
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.0412555495194207, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.04)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: forward, reward: -0.0690491153867
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': -0.06904911538665848, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded -0.07)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: forward, reward: 0.527497448676
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 0.5274974486762084, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 0.53)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 1.11508401332
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.1150840133211228, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 1.12)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.32400887957
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.3240088795724034, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.32)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 2.16644418743
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 2.1664441874308538, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.17)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: -0.303224377268
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', 'left'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': -0.30322437726772944, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', 'left')
Agent drove right instead of forward. (rewarded -0.30)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 0.360451768086
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.36045176808585877, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 0.36)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 157
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (3, 3), deadline = 20
Simulating trial. . .
epsilon = 0.2080; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: forward, reward: 0.923257823269
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 0.9232578232690625, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'left')
Agent drove forward instead of right. (rewarded 0.92)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.43462255062
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.4346225506192445, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.43)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.16692018085
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.1669201808489231, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.17)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: 2.43888302634
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.438883026343153, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.44)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: -39.847685571
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': -39.847685571007325, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.85)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.92612439274
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.9261243927436793, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.93)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.43056629521
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.4305662952065, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.43)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 1.5595632188
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.5595632188028778, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.56)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 1.55650643296
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.5565064329587794, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.56)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 1.06136328349
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.0613632834865474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 1.06)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: left, reward: 1.66048113147
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.6604811314726777, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 1.66)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 158
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (8, 6), deadline = 30
Simulating trial. . .
epsilon = 0.2060; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 2.43471315607
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.4347131560739816, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.43)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 1.14897268862
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.1489726886243758, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.15)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 1.87852074723
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.8785207472349992, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.88)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 1.13248646206
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.1324864620567712, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.13)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 2.43212834067
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.4321283406698235, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.43)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: forward, reward: 1.46179511272
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.4617951127174629, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.46)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 2.58208216038
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.582082160383772, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.58)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: 1.7560182138
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 1.756018213801707, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.76)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 0.976086120739
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 8, 'action': None, 'reward': 0.976086120739132, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.98)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 2.53584234502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 9, 'action': None, 'reward': 2.5358423450209235, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.54)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 2.37993104385
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 2.3799310438497487, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.38)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: forward, reward: 1.45524894059
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.455248940592697, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.46)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: forward, reward: 0.284593866694
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': 0.28459386669418285, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.28)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: right, reward: 1.89992567812
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'forward'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 1.8999256781187073, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.90)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: None, reward: 1.2410227788
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.241022778803746, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.24)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: right, reward: 1.76127478186
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.7612747818593413, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.76)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.7937724733
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 14, 't': 16, 'action': None, 'reward': 1.79377247329987, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.79)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 2.33883592985
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 17, 'action': None, 'reward': 2.338835929854576, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.34)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.68462008046
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.684620080458142, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.68)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 2.21921364672
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 11, 't': 19, 'action': None, 'reward': 2.219213646715061, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.22)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.71798272309
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 20, 'action': None, 'reward': 1.7179827230893276, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.72)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: left, reward: 2.36731403973
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'left', 'reward': 2.367314039727791, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.37)
27% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 159
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (8, 3), deadline = 25
Simulating trial. . .
epsilon = 0.2039; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 2.15971587288
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.159715872881936, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.16)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.5661345454
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.5661345454030404, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.57)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 2.1967976801
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.196797680101178, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.20)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 1.21616158115
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.216161581154794, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.22)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 2.64716707322
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.6471670732220174, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.65)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: 2.90076141764
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.9007614176366645, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.90)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 2.63373790681
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.6337379068101443, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.63)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: left, reward: 1.28550421442
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 1.2855042144160909, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 1.29)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: forward, reward: 1.06792950323
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 1.0679295032251417, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.07)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: forward, reward: 2.3421033282
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.3421033282011505, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.34)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 160
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (5, 3), deadline = 20
Simulating trial. . .
epsilon = 0.2019; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: forward, reward: 1.82495054494
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.8249505449430392, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove forward instead of left. (rewarded 1.82)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 1.91914670563
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.919146705633172, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 1.92)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 1.32909685797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.329096857973805, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.33)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: forward, reward: -39.4340457577
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': -39.434045757733685, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.43)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 1.98606996115
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.9860699611459536, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.99)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 2.29470601305
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.2947060130502313, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.29)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: left, reward: 2.29506052034
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.2950605203401855, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.30)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 2.16940834948
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.1694083494777257, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.17)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 0.890040854511
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 0.890040854511174, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.89)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 0.811974017296
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.8119740172957417, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 0.81)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 1.19143076452
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.1914307645153728, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.19)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: left, reward: 2.00938871338
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 2.009388713379378, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.01)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: forward, reward: 1.11658637037
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 1.1165863703686931, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.12)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: 1.30278377607
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.3027837760736898, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.30)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 161
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (6, 6), deadline = 25
Simulating trial. . .
epsilon = 0.1999; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 1.28268460177
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.282684601768548, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.28)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 2.1714665979
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.1714665978951677, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.17)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 2.30280415682
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.3028041568198976, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.30)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 2.92005048659
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.920050486589021, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 2.92)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 1.21290483235
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.2129048323530296, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.21)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 0.50625176479
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.506251764789981, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.51)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 1.42605708257
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.4260570825724794, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.43)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 1.00026552701
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.0002655270099838, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent drove right instead of left. (rewarded 1.00)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.67707825512
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.6770782551162684, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.68)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.32150769223
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.3215076922321392, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.32)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 0.378498005071
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 0.3784980050711292, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.38)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 0.239526156993
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 0.23952615699300572, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove forward instead of left. (rewarded 0.24)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 1.16785771168
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.1678577116806863, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 1.17)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 2.41805003136
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 2.418050031356456, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.42)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 1.28180851577
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.2818085157669603, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.28)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 2.09759168862
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 2.0975916886184773, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.10)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: left, reward: -0.215607939855
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 9, 't': 16, 'action': 'left', 'reward': -0.2156079398547951, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded -0.22)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 0.650084708947
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 0.6500847089467898, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.65)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: left, reward: 2.15047491162
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 7, 't': 18, 'action': 'left', 'reward': 2.150474911624531, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 2.15)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: 2.33808219421
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 6, 't': 19, 'action': None, 'reward': 2.3380821942064687, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.34)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: 1.77636059026
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.7763605902562032, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.78)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: forward, reward: 0.955074276716
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 0.955074276716056, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.96)
12% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 162
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (4, 6), deadline = 20
Simulating trial. . .
epsilon = 0.1979; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 1.76700833414
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.767008334143472, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.77)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 1.96689083224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.9668908322404546, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.97)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 2.55450951829
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.55450951828551, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.55)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 1.25420454057
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'forward'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.2542045405702442, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'forward')
Agent followed the waypoint right. (rewarded 1.25)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: 1.47447446215
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.4744744621511934, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.47)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.88434776174
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.8843477617444078, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.88)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: 1.49024382614
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.4902438261424986, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.49)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: left, reward: 1.56959625133
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.5695962513250432, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.57)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 163
\-------------------------
Environment.reset(): Trial set up with start = (2, 6), destination = (6, 4), deadline = 30
Simulating trial. . .
epsilon = 0.1959; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1959; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1959; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1959; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 1.38448897638
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'right'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.3844889763813604, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'right')
Agent followed the waypoint right. (rewarded 1.38)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 0.671657392184
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 0.6716573921844128, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove right instead of forward. (rewarded 0.67)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 1.86270697044
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 1.8627069704429609, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.86)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 2.51579974097
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.515799740972173, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.52)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.51168610047
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.511686100471623, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.51)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: left, reward: 1.84060271513
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 1.8406027151283513, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.84)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: left, reward: 1.77187146285
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 1.771871462853309, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.77)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.31789009913
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 2.317890099130966, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.32)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.69395778646
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.6939577864571806, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.69)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.11088153321
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.1108815332073072, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.11)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.00351617584
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.0035161758440434, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.00)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 1.21418732588
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.2141873258761362, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.21)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 2.48441426384
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.4844142638360367, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.48)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 1.63276995751
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.6327699575136152, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 1.63)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 0.878775783247
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'right'), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 0.8787757832465242, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'right')
Agent drove right instead of forward. (rewarded 0.88)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 1.63157189609
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.6315718960941554, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.63)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 0.984318706961
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 16, 'action': None, 'reward': 0.9843187069612163, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.98)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: left, reward: 1.04658957468
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 17, 'action': 'left', 'reward': 1.0465895746823226, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.05)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 0.456237217911
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 0.4562372179114542, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 0.46)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: 2.53255835935
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 2.532558359350445, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.53)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: 2.08760222961
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 2.08760222961353, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.09)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 2.4543183269
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 9, 't': 21, 'action': None, 'reward': 2.4543183269047084, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.45)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 0.966550059517
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'right', 'right'), 'deadline': 8, 't': 22, 'action': None, 'reward': 0.9665500595174568, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', 'right')
Agent properly idled at a red light. (rewarded 0.97)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 0.866295820393
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 23, 'action': None, 'reward': 0.8662958203932438, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.87)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 0.439726831756
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 0.43972683175594063, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 0.44)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 0.525328685133
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 5, 't': 25, 'action': None, 'reward': 0.5253286851331143, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 0.53)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: forward, reward: -9.21132092394
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'left'), 'deadline': 4, 't': 26, 'action': 'forward', 'reward': -9.211320923935123, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -9.21)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 1.51918969043
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 3, 't': 27, 'action': None, 'reward': 1.5191896904303843, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.52)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: left, reward: 1.16718432532
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 2, 't': 28, 'action': 'left', 'reward': 1.1671843253163965, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.17)
3% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 164
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (6, 6), deadline = 25
Simulating trial. . .
epsilon = 0.1940; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 1.60165583165
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.6016558316505243, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 1.60)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: left, reward: 1.85771541994
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', 'left'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 1.8577154199437744, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', 'left')
Agent drove left instead of forward. (rewarded 1.86)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 1.45027600213
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.4502760021338086, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.45)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.56108853811
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.5610885381052404, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.56)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.8473775855
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.8473775854967007, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.85)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.4606843541
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.4606843540963865, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.46)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: 2.84239270263
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.84239270262566, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 2.84)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 1.24811331826
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.2481133182641686, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.25)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: right, reward: 0.406718366274
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.40671836627361413, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.41)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: right, reward: 1.20349180906
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.203491809055784, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.20)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 1.63795060008
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.6379506000762478, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.64)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: right, reward: 2.15614062665
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 2.156140626651988, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.16)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 2.26931559108
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.2693155910758533, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 2.27)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 1.19063802172
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 1.1906380217223367, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.19)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: -10.6404154597
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': -10.64041545974639, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.64)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: left, reward: 1.54323932977
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 1.5432393297745381, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.54)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: 0.87076427381
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 0.8707642738099469, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.87)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: None, reward: 2.37979829105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 8, 't': 17, 'action': None, 'reward': 2.3797982910545095, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.38)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: right, reward: 1.99322108
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 1.9932210800024206, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.99)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 1.90791476453
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 6, 't': 19, 'action': None, 'reward': 1.9079147645347134, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.91)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: right, reward: 0.818495057836
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 0.8184950578360881, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 0.82)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: 1.60849637438
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 4, 't': 21, 'action': None, 'reward': 1.6084963743789384, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.61)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: right, reward: -0.708250677579
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'right'), 'deadline': 3, 't': 22, 'action': 'right', 'reward': -0.7082506775787503, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'right')
Agent drove right instead of forward. (rewarded -0.71)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: left, reward: -9.59559832233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 2, 't': 23, 'action': 'left', 'reward': -9.595598322325628, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.60)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 1.73837922357
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'forward'), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.7383792235665163, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'forward', 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.74)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 165
\-------------------------
Environment.reset(): Trial set up with start = (7, 4), destination = (5, 6), deadline = 20
Simulating trial. . .
epsilon = 0.1920; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 1.63140737099
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.6314073709902925, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.63)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: right, reward: 2.56555531915
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.5655553191524754, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.57)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: left, reward: 1.76847940162
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.7684794016232446, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.77)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: left, reward: -20.0121238495
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -20.01212384951936, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.01)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: right, reward: 2.84819209256
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.8481920925603648, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.85)
75% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 166
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (3, 5), deadline = 25
Simulating trial. . .
epsilon = 0.1901; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1901; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1901; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: forward, reward: -40.1150048065
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -40.11500480648622, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.12)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 1.37302928582
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.3730292858239463, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.37)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 2.02978252698
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.0297825269799636, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.03)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 2.62914637821
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 2.629146378211374, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.63)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: -20.5707048291
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'right'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': -20.570704829112422, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'right')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.57)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: -10.5870074468
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': -10.58700744678538, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.59)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.60617024198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.6061702419840547, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.61)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: 1.78346370453
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'right'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.7834637045348252, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'right')
Agent drove right instead of forward. (rewarded 1.78)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 2.86946629141
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.869466291412767, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.87)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 1.43026688508
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.4302668850819542, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.43)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 2.2398822695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.23988226949581, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.24)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: left, reward: 2.08011298593
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 2.080112985930316, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.08)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 2.67239449446
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.672394494461825, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.67)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 1.77247773574
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.7724777357420567, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.77)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 2.53286097456
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.532860974557661, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.53)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: -20.6768238861
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': -20.67682388612223, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.68)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: 0.827336344002
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 0.8273363440018069, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.83)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 1.26042315336
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.2604231533558292, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.26)
28% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 167
\-------------------------
Environment.reset(): Trial set up with start = (2, 6), destination = (7, 4), deadline = 25
Simulating trial. . .
epsilon = 0.1882; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1882; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1882; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1882; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: -4.15559986443
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'right'), 'deadline': 25, 't': 0, 'action': None, 'reward': -4.155599864425324, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.16)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 1.58049964637
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'right'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.5804996463718661, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'right')
Agent drove right instead of left. (rewarded 1.58)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: right, reward: 2.67808359559
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 2.6780835955936113, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.68)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: left, reward: 0.995657156316
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 0.9956571563160301, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent drove left instead of forward. (rewarded 1.00)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 2.44373419542
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.4437341954205376, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.44)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: 0.418525046145
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 0.41852504614463826, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove left instead of right. (rewarded 0.42)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 1.29446274225
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.294462742254422, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.29)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 1.47017334796
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.4701733479590815, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.47)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.81006476724
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.8100647672362007, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.81)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.10157084738
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.101570847375087, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.10)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 1.68253185336
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 1.6825318533616438, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.68)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: -9.02326873623
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'right'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': -9.023268736230257, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'right')
Agent attempted driving forward through a red light. (rewarded -9.02)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 2.38177302485
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'right'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.3817730248487186, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'right')
Agent properly idled at a red light. (rewarded 2.38)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 1.44319569058
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 1.4431956905781107, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 1.44)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: left, reward: 1.77540010366
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 1.7754001036562892, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.78)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 168
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (6, 4), deadline = 20
Simulating trial. . .
epsilon = 0.1864; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1864; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1864; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: forward, reward: 0.433831504563
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 0.4338315045633956, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded 0.43)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.18970229031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.1897022903062375, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.19)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.2960155518
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.2960155518025305, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.30)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.24525826268
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.2452582626817794, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.25)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.65315528185
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.653155281846472, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.65)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 0.965580678543
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 0.9655806785428658, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.97)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: 1.95900809374
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.9590080937403036, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.96)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 1.00971940128
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.0097194012777042, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.01)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.08778338193
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.087783381930768, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.09)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: 2.28341993067
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 2.2834199306671135, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 2.28)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: forward, reward: 1.3198017489
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.31980174889865, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.32)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: forward, reward: 1.25779434873
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.2577943487265164, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.26)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 169
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (1, 6), deadline = 35
Simulating trial. . .
epsilon = 0.1845; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 0.0773595685167
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'right'), 'deadline': 35, 't': 0, 'action': 'right', 'reward': 0.07735956851671921, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'right')
Agent drove right instead of left. (rewarded 0.08)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.15910409729
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 34, 't': 1, 'action': None, 'reward': 1.1591040972895126, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.16)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.42735102603
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 33, 't': 2, 'action': None, 'reward': 1.4273510260256863, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.43)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 0.988789838506
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 32, 't': 3, 'action': 'right', 'reward': 0.9887898385060794, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.99)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 1.95646840921
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 31, 't': 4, 'action': None, 'reward': 1.9564684092102742, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.96)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 1.48646373648
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 30, 't': 5, 'action': None, 'reward': 1.4864637364838291, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.49)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: left, reward: -10.7648082658
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 29, 't': 6, 'action': 'left', 'reward': -10.764808265819507, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent attempted driving left through a red light. (rewarded -10.76)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: left, reward: 1.48598917923
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 28, 't': 7, 'action': 'left', 'reward': 1.4859891792303919, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.49)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: left, reward: 1.82529299529
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 27, 't': 8, 'action': 'left', 'reward': 1.8252929952888803, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.83)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 2.21399217163
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 26, 't': 9, 'action': 'right', 'reward': 2.2139921716267628, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.21)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 2.09763565408
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 10, 'action': 'forward', 'reward': 2.0976356540751917, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.10)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 0.998802845159
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 11, 'action': 'right', 'reward': 0.9988028451589517, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.00)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 0.886042786426
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 23, 't': 12, 'action': 'right', 'reward': 0.886042786425829, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.89)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 2.71707314019
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 22, 't': 13, 'action': 'left', 'reward': 2.717073140191308, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.72)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: forward, reward: 0.993004085417
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', None), 'deadline': 21, 't': 14, 'action': 'forward', 'reward': 0.9930040854168475, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', None)
Agent drove forward instead of left. (rewarded 0.99)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 1.9054529167
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 20, 't': 15, 'action': None, 'reward': 1.905452916699381, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.91)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: left, reward: -10.9861645501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 19, 't': 16, 'action': 'left', 'reward': -10.986164550139907, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.99)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: forward, reward: 0.926611348623
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 18, 't': 17, 'action': 'forward', 'reward': 0.9266113486230608, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove forward instead of left. (rewarded 0.93)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: None, reward: 2.50449208035
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 18, 'action': None, 'reward': 2.504492080351301, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.50)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: left, reward: -10.2936642574
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 19, 'action': 'left', 'reward': -10.293664257424687, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.29)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: None, reward: 2.58818265575
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 15, 't': 20, 'action': None, 'reward': 2.588182655753952, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.59)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: forward, reward: 0.990829486988
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 14, 't': 21, 'action': 'forward', 'reward': 0.9908294869879715, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 0.99)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 1.64563553602
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 13, 't': 22, 'action': None, 'reward': 1.6456355360163912, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.65)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 0.87457068674
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 12, 't': 23, 'action': None, 'reward': 0.8745706867397782, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.87)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 1.66307255618
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 11, 't': 24, 'action': None, 'reward': 1.663072556184608, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.66)
29% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 0.582078433793
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 25, 'action': None, 'reward': 0.582078433793181, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.58)
26% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: -0.0618745397321
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 26, 'action': 'forward', 'reward': -0.06187453973209367, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded -0.06)
23% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 0.792498826726
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 27, 'action': None, 'reward': 0.7924988267262754, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.79)
20% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 1.86566591452
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 7, 't': 28, 'action': None, 'reward': 1.8656659145155545, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.87)
17% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: -0.342310819515
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 6, 't': 29, 'action': 'right', 'reward': -0.3423108195154997, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent drove right instead of left. (rewarded -0.34)
14% of time remaining to reach destination.
/-------------------
| Step 30 Results
\-------------------
Environment.step(): t = 30
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 0.903290115835
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 5, 't': 30, 'action': None, 'reward': 0.9032901158346225, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 0.90)
11% of time remaining to reach destination.
/-------------------
| Step 31 Results
\-------------------
Environment.step(): t = 31
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: left, reward: 1.5419310943
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 31, 'action': 'left', 'reward': 1.5419310942981754, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.54)
9% of time remaining to reach destination.
/-------------------
| Step 32 Results
\-------------------
Environment.step(): t = 32
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 1.29196052898
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 3, 't': 32, 'action': None, 'reward': 1.2919605289815077, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.29)
6% of time remaining to reach destination.
/-------------------
| Step 33 Results
\-------------------
Environment.step(): t = 33
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: left, reward: 0.387632170584
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 33, 'action': 'left', 'reward': 0.38763217058389965, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.39)
3% of time remaining to reach destination.
/-------------------
| Step 34 Results
\-------------------
Environment.step(): t = 34
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 0.440826350463
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 1, 't': 34, 'action': None, 'reward': 0.44082635046253293, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.44)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 170
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (5, 6), deadline = 20
Simulating trial. . .
epsilon = 0.1827; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1827; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1827; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 0.193881935211
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 0.1938819352110821, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.19)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 1.52059342123
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.5205934212322314, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.52)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 1.42188592925
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.4218859292537187, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.42)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 0.837897415648
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 0.8378974156475937, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.84)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 1.61399194033
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.6139919403341887, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 1.61)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 0.574886421357
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.5748864213571243, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'left')
Agent drove right instead of left. (rewarded 0.57)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 2.79058992312
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.7905899231246227, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.79)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.82012848543
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.8201284854256132, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.82)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 2.52314626197
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'left', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.5231462619678258, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'left', None)
Agent followed the waypoint right. (rewarded 2.52)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.40518444523
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.4051844452257758, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.41)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.03469968804
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.034699688042924, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.03)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: 2.43227470447
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 2.4322747044655806, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.43)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 2.49969668229
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 2.499696682285323, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.50)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: left, reward: 2.49165763833
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 2.4916576383284443, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.49)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 171
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (6, 6), deadline = 25
Simulating trial. . .
epsilon = 0.1809; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1809; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1809; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1809; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1809; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1809; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1809; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 0.0943556967446
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 0.09435569674460731, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded 0.09)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.95427124546
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 1.9542712454560922, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 1.95)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: 1.768013204
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 1.7680132040012366, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.77)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: left, reward: 0.740109537737
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 0.7401095377372642, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.74)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 2.273655621
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.2736556210021677, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.27)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: right, reward: 2.77583884148
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 2.7758388414791475, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.78)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: 1.61356513648
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.6135651364823076, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'forward')
Agent followed the waypoint forward. (rewarded 1.61)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 0.365320309307
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'right'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.36532030930728854, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'right')
Agent drove right instead of left. (rewarded 0.37)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 1.39117753639
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.3911775363895653, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.39)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 2.36687030541
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 2.366870305413876, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 2.37)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 1.22838639834
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.2283863983358154, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.23)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 0.864362144188
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': 0.8643621441881262, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.86)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 1.09585945078
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.0958594507764434, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.10)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: -5.39058978163
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': -5.390589781633425, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.39)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: right, reward: 2.08181342679
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 2.081813426791262, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.08)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: 1.02200137358
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 1.0220013735755142, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.02)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 1.55509150492
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 1.5550915049247438, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.56)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 2.32931709327
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 8, 't': 17, 'action': None, 'reward': 2.3293170932658662, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.33)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 2.32112668077
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 2.321126680767037, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.32)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 1.46841411026
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 1.4684141102563397, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 1.47)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 1.15816745819
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 1.1581674581923749, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.16)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: 1.16195948218
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 1.1619594821801729, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.16)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 2.05227988181
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 2.0522798818061903, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.05)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: -0.238838853149
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 2, 't': 23, 'action': 'right', 'reward': -0.23883885314914943, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent drove right instead of forward. (rewarded -0.24)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 0.763620394233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.7636203942333639, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 0.76)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 172
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (5, 7), deadline = 20
Simulating trial. . .
epsilon = 0.1791; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 1.88847008993
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.888470089926849, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.89)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 2.7262764683
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.7262764682971263, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.73)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 2.23645932002
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 2.2364593200177447, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.24)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.92199337344
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.9219933734444066, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.92)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 1.04274544847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.0427454484704555, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.04)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.36954469069
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.369544690686472, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.37)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 1.952149395
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.9521493950044277, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.95)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: 0.962254863514
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 0.9622548635139019, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 0.96)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 173
\-------------------------
Environment.reset(): Trial set up with start = (4, 3), destination = (8, 7), deadline = 30
Simulating trial. . .
epsilon = 0.1773; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 2.37237898411
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.3723789841148086, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.37)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.79167641482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.7916764148157909, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.79)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: left, reward: -10.293657698
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': 'left', 'reward': -10.29365769801664, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.29)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: -10.8137382909
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': -10.813738290933868, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.81)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.21422248142
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.2142224814175466, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.21)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: left, reward: -0.00218923977581
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 25, 't': 5, 'action': 'left', 'reward': -0.0021892397758122017, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded -0.00)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 1.42414950323
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 1.424149503229995, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.42)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 1.69603496684
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.6960349668360968, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.70)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 2.61838881664
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.6183888166444245, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.62)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 1.04980917458
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.0498091745753575, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.05)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 1.36125187468
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 1.3612518746795863, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.36)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.73929578842
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.739295788417877, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.74)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 0.946215671279
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 0.9462156712786475, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.95)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: left, reward: 2.05885492729
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 2.0588549272895356, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.06)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 1.17988249889
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.1798824988867178, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.18)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: forward, reward: 0.778412852204
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': 0.7784128522035776, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 0.78)
47% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 174
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (7, 7), deadline = 20
Simulating trial. . .
epsilon = 0.1755; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 1.05923301241
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.0592330124052256, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.06)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 2.4109928023
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.410992802295839, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.41)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 1.75829548803
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.758295488033907, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.76)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: -9.22342642149
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -9.223426421490153, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.22)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 1.29964755663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.299647556629692, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.30)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 1.67817998123
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'right'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.6781799812282527, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.68)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 1.29884602063
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.2988460206324521, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 1.30)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: left, reward: 1.21178144271
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.211781442706209, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 1.21)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 2.15860232716
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 2.158602327160013, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 2.16)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.62063340142
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.620633401416764, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.62)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 1.65983388165
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.6598338816476708, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 1.66)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.12616419609
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.1261641960852218, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 1.13)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.55231790602
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.552317906018739, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.55)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: 1.89767699275
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.8976769927489052, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.90)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: left, reward: 1.9185176181
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 1.9185176180955703, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.92)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: -0.354746620271
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': -0.3547466202710263, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded -0.35)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: right, reward: 0.911480633771
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.9114806337709662, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent drove right instead of left. (rewarded 0.91)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 1.51605244952
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 1.5160524495201682, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.52)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 0.616560086706
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.6165600867059671, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 0.62)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: -9.31252117474
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -9.312521174741516, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.31)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 175
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (2, 6), deadline = 20
Simulating trial. . .
epsilon = 0.1738; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1738; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1738; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: -9.64872417583
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -9.648724175831749, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.65)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 1.87896072749
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.8789607274894229, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.88)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 1.76165589618
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.7616558961841444, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.76)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: left, reward: -10.9244591457
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -10.924459145719293, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.92)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 2.42068705538
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.4206870553805278, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.42)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: left, reward: 1.83086386601
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.8308638660130654, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.83)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 1.42792496537
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.4279249653653747, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.43)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 1.60178949291
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.6017894929124596, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.60)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 176
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (3, 7), deadline = 20
Simulating trial. . .
epsilon = 0.1720; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 0.529015763905
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.5290157639052704, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'left')
Agent drove right instead of left. (rewarded 0.53)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 1.99318619833
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.9931861983303492, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.99)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: 0.919294359162
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 0.9192943591615249, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.92)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: -10.3625115691
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': -10.36251156907933, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.36)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 2.28042520941
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.2804252094136133, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 2.28)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: 0.960449148415
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 0.9604491484150817, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 0.96)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 2.11186895213
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.1118689521298997, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.11)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: left, reward: 2.20184463185
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 2.201844631851685, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.20)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: -0.0987421849956
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', 'right'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': -0.09874218499563792, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', 'right')
Agent drove right instead of forward. (rewarded -0.10)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 0.904367223647
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.904367223647371, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 0.90)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 0.960946455592
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 0.9609464555922724, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.96)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 1.99953831144
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.999538311444379, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.00)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 2.11722368841
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.1172236884065994, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.12)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: left, reward: 2.01763275032
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 2.0176327503158413, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.02)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: left, reward: -19.3412139557
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -19.341213955734798, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.34)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: left, reward: 2.34008155881
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 2.3400815588144703, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.34)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: right, reward: 1.63461485954
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.6346148595382655, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.63)
15% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 177
\-------------------------
Environment.reset(): Trial set up with start = (2, 7), destination = (5, 3), deadline = 25
Simulating trial. . .
epsilon = 0.1703; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1703; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1703; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1703; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: left, reward: 1.07218055104
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 1.0721805510370892, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.07)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: left, reward: 1.13644338675
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 1.136443386750185, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.14)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 1.24292811418
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.2429281141804216, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 1.24)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: forward, reward: 1.69163740672
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.691637406718178, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 1.69)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: left, reward: -39.8124171965
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 21, 't': 4, 'action': 'left', 'reward': -39.812417196463734, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.81)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: forward, reward: 1.44239957127
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.442399571267559, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.44)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: forward, reward: -0.00145032342296
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -0.0014503234229615858, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove forward instead of left. (rewarded -0.00)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: forward, reward: 0.815788550597
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 0.81578855059723, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded 0.82)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: 2.6103011652
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.6103011652021664, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.61)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: 2.18676348188
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.1867634818798427, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.19)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: 1.06353043147
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.0635304314702791, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.06)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: 0.891955434133
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 0.8919554341326743, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 0.89)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.68405923389
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.684059233888087, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.68)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: left, reward: 0.714508181863
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 0.7145081818631569, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.71)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 1.66784733762
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.6678473376153022, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.67)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 1.94603211153
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.9460321115344674, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.95)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 0.653218064506
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 9, 't': 16, 'action': None, 'reward': 0.6532180645062996, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.65)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 1.317033449
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.317033449001603, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.32)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 1.9729132985
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.9729132985028508, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.97)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: forward, reward: 2.42398870083
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 2.423988700826106, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.42)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: forward, reward: 1.06722036022
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 5, 't': 20, 'action': 'forward', 'reward': 1.0672203602211952, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.07)
16% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 178
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (4, 7), deadline = 35
Simulating trial. . .
epsilon = 0.1686; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1686; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: right, reward: 1.18633592508
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 35, 't': 0, 'action': 'right', 'reward': 1.1863359250791856, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove right instead of left. (rewarded 1.19)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 1.66018021437
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 34, 't': 1, 'action': None, 'reward': 1.6601802143718645, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.66)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 1.38323291459
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 33, 't': 2, 'action': None, 'reward': 1.3832329145853839, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.38)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 2.44391844821
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 32, 't': 3, 'action': None, 'reward': 2.4439184482087546, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.44)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 2.82645376418
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 31, 't': 4, 'action': None, 'reward': 2.82645376417699, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.83)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 1.95338533557
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 30, 't': 5, 'action': None, 'reward': 1.9533853355720237, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.95)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: 1.25879774461
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 29, 't': 6, 'action': 'forward', 'reward': 1.2587977446078047, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.26)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: 1.25762776133
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 28, 't': 7, 'action': 'right', 'reward': 1.2576277613254034, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.26)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: 1.28292600387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 27, 't': 8, 'action': None, 'reward': 1.2829260038738513, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.28)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: 1.52412041839
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 26, 't': 9, 'action': None, 'reward': 1.5241204183919441, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.52)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: 1.60959234963
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 10, 'action': 'left', 'reward': 1.6095923496262239, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.61)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: 1.01741485941
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 24, 't': 11, 'action': 'forward', 'reward': 1.0174148594126688, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.02)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 1.02372817218
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 23, 't': 12, 'action': 'right', 'reward': 1.023728172179677, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.02)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 2.6249536899
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 22, 't': 13, 'action': None, 'reward': 2.6249536898952988, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.62)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 1.32839117172
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 21, 't': 14, 'action': None, 'reward': 1.3283911717215315, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 1.33)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 1.09408060445
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 15, 'action': None, 'reward': 1.0940806044504994, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.09)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: left, reward: -0.00219573728342
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 19, 't': 16, 'action': 'left', 'reward': -0.002195737283421706, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded -0.00)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 0.963070340311
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'forward'), 'deadline': 18, 't': 17, 'action': 'forward', 'reward': 0.9630703403111404, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'forward')
Agent drove forward instead of right. (rewarded 0.96)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 2.22743915994
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 17, 't': 18, 'action': 'right', 'reward': 2.227439159939548, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.23)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 0.867469914247
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 16, 't': 19, 'action': 'right', 'reward': 0.8674699142470446, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 0.87)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: 1.51567187274
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 20, 'action': 'forward', 'reward': 1.5156718727433358, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.52)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 179
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (4, 3), deadline = 20
Simulating trial. . .
epsilon = 0.1670; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.54184073437
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.5418407343714082, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.54)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 2.19772218965
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.1977221896461794, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.20)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.73455725338
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.7345572533780298, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.73)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.56822187933
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.568221879333279, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.57)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.2625161184
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.262516118395017, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.26)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.31900729608
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.3190072960849915, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.32)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: forward, reward: 2.36911700542
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 2.3691170054248474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.37)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: 1.28567506257
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.285675062570927, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.29)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: left, reward: 1.47731909305
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.4773190930530777, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', None)
Agent drove left instead of forward. (rewarded 1.48)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 0.941358846522
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.9413588465223619, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.94)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 180
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (4, 5), deadline = 35
Simulating trial. . .
epsilon = 0.1653; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1653; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 2.45105435155
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 35, 't': 0, 'action': 'forward', 'reward': 2.4510543515500247, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.45)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: left, reward: 0.576453519532
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', 'left'), 'deadline': 34, 't': 1, 'action': 'left', 'reward': 0.5764535195323837, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', 'left')
Agent drove left instead of forward. (rewarded 0.58)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 1.25934749403
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 33, 't': 2, 'action': None, 'reward': 1.2593474940304288, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.26)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 1.04112684317
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 32, 't': 3, 'action': 'right', 'reward': 1.0411268431651595, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.04)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: 2.87938325264
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 31, 't': 4, 'action': 'forward', 'reward': 2.879383252635284, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.88)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: 1.85916981757
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 30, 't': 5, 'action': 'forward', 'reward': 1.8591698175733642, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.86)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: left, reward: 2.74258818599
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 29, 't': 6, 'action': 'left', 'reward': 2.7425881859879935, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 2.74)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 0.262296192094
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 28, 't': 7, 'action': 'right', 'reward': 0.2622961920936472, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded 0.26)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: 1.37057181192
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 27, 't': 8, 'action': 'forward', 'reward': 1.3705718119181292, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.37)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.03133106373
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 26, 't': 9, 'action': None, 'reward': 1.0313310637295974, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.03)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 2.22931383007
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'right'), 'deadline': 25, 't': 10, 'action': None, 'reward': 2.2293138300748163, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'right')
Agent properly idled at a red light. (rewarded 2.23)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 1.68250624382
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 24, 't': 11, 'action': 'forward', 'reward': 1.6825062438164817, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.68)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 1.67621188413
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 12, 'action': None, 'reward': 1.6762118841321823, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.68)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 0.183290964268
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 22, 't': 13, 'action': 'right', 'reward': 0.1832909642682018, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent drove right instead of left. (rewarded 0.18)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: 0.225356742262
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'forward'), 'deadline': 21, 't': 14, 'action': 'forward', 'reward': 0.22535674226172808, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'forward')
Agent drove forward instead of right. (rewarded 0.23)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: 1.51987662519
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 20, 't': 15, 'action': 'left', 'reward': 1.5198766251939544, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove left instead of right. (rewarded 1.52)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: right, reward: 0.42317038566
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 19, 't': 16, 'action': 'right', 'reward': 0.4231703856597283, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 0.42)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: -5.78909777308
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 18, 't': 17, 'action': None, 'reward': -5.789097773076302, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.79)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: 2.72957300001
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 17, 't': 18, 'action': 'left', 'reward': 2.729573000013298, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.73)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 1.73816343948
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 19, 'action': 'forward', 'reward': 1.7381634394795813, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.74)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 1.94073805457
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 20, 'action': 'forward', 'reward': 1.9407380545730106, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.94)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 2.162314946
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 14, 't': 21, 'action': 'forward', 'reward': 2.162314945997478, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.16)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 0.923935234442
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 13, 't': 22, 'action': 'right', 'reward': 0.9239352344416385, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.92)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 2.26938721201
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 23, 'action': None, 'reward': 2.2693872120115435, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.27)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 1.19407633544
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 11, 't': 24, 'action': 'right', 'reward': 1.1940763354376775, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.19)
29% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: -10.8169270682
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 10, 't': 25, 'action': 'left', 'reward': -10.81692706816924, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.82)
26% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: -39.4185062488
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 9, 't': 26, 'action': 'forward', 'reward': -39.418506248800384, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.42)
23% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: right, reward: 1.14297506394
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 8, 't': 27, 'action': 'right', 'reward': 1.142975063936865, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 1.14)
20% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 2.12549706937
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 7, 't': 28, 'action': 'right', 'reward': 2.125497069369531, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.13)
17% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 0.936087046917
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 6, 't': 29, 'action': 'right', 'reward': 0.9360870469166511, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.94)
14% of time remaining to reach destination.
/-------------------
| Step 30 Results
\-------------------
Environment.step(): t = 30
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 1.88751626847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 5, 't': 30, 'action': None, 'reward': 1.8875162684662057, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.89)
11% of time remaining to reach destination.
/-------------------
| Step 31 Results
\-------------------
Environment.step(): t = 31
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: -0.0108722952157
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 4, 't': 31, 'action': 'right', 'reward': -0.01087229521566202, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded -0.01)
9% of time remaining to reach destination.
/-------------------
| Step 32 Results
\-------------------
Environment.step(): t = 32
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: left, reward: 1.12003092917
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 3, 't': 32, 'action': 'left', 'reward': 1.1200309291733137, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.12)
6% of time remaining to reach destination.
/-------------------
| Step 33 Results
\-------------------
Environment.step(): t = 33
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: forward, reward: -0.0213967695663
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 2, 't': 33, 'action': 'forward', 'reward': -0.02139676956630976, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent drove forward instead of left. (rewarded -0.02)
3% of time remaining to reach destination.
/-------------------
| Step 34 Results
\-------------------
Environment.step(): t = 34
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: left, reward: 2.00392199633
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 1, 't': 34, 'action': 'left', 'reward': 2.003921996325357, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.00)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 181
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (1, 5), deadline = 25
Simulating trial. . .
epsilon = 0.1637; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: 1.15124740195
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 1.1512474019521808, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.15)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.23412032486
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.2341203248618757, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.23)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: right, reward: 0.904279097174
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'forward'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 0.9042790971740733, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.90)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: left, reward: 2.72352533888
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 2.723525338884564, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.72)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 2.78878249861
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.788782498607796, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.79)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 2.11819332476
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.1181933247630154, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.12)
76% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 182
\-------------------------
Environment.reset(): Trial set up with start = (7, 3), destination = (1, 6), deadline = 25
Simulating trial. . .
epsilon = 0.1620; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1620; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1620; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1620; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 2.7784999137
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.778499913698491, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.78)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 1.47303104058
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.4730310405775484, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.47)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 1.0029608783
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.0029608782971864, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.00)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 1.58130379382
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.5813037938166778, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.58)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 1.88917018586
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.889170185857894, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.89)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 2.11949444738
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 2.1194944473847004, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.12)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 0.920548712217
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.9205487122166074, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.92)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 2.42552198931
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 2.425521989306535, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.43)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 1.80128323981
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 1.8012832398063297, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.80)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 1.18120418738
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.1812041873821877, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.18)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: forward, reward: 0.38897985511
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 0.38897985510995625, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.39)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 1.95447657368
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.9544765736809115, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.95)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 1.61915766777
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.6191576677724502, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.62)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: 1.31478030028
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 1.3147803002841298, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.31)
44% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 183
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (7, 5), deadline = 20
Simulating trial. . .
epsilon = 0.1604; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1604; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1604; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: -5.93614985894
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'right'), 'deadline': 20, 't': 0, 'action': None, 'reward': -5.936149858944663, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.94)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: 1.93931864277
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.9393186427654028, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.94)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 1.65170941837
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.6517094183739802, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 1.65)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 2.45833404745
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.458334047451721, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.46)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 1.33471899481
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.334718994811835, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.33)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: left, reward: 0.633737047351
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 0.6337370473510218, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 0.63)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.2785139814
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.2785139813973847, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.28)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 2.15566082599
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 2.15566082599369, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.16)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 184
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (5, 7), deadline = 20
Simulating trial. . .
epsilon = 0.1588; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1588; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1588; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 2.86469106399
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'right'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.864691063992872, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 2.86)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 0.852107911877
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.8521079118767508, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.85)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: forward, reward: 1.10415092683
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.1041509268310312, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 1.10)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: 1.31924339665
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.319243396650873, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.32)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: left, reward: -39.9458599271
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -39.945859927056155, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.95)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: 2.52226855222
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.522268552222262, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.52)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: 0.942702762351
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 0.9427027623511739, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'left')
Agent followed the waypoint left. (rewarded 0.94)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.51436820784
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.5143682078387393, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.51)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 1.83570946265
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.8357094626528223, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.84)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: forward, reward: 1.9988933199
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.998893319901669, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.00)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 185
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (4, 5), deadline = 20
Simulating trial. . .
epsilon = 0.1572; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: right, reward: 1.56368611135
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.5636861113490457, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 1.56)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: -19.69506018
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'forward'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -19.695060180012923, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.70)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: 1.02147563475
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'right'), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.0214756347515326, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'right')
Agent followed the waypoint forward. (rewarded 1.02)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 1.64541739318
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.6454173931809684, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 1.65)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: left, reward: 2.58182139558
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 2.581821395579073, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 2.58)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: left, reward: 2.57354573173
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.57354573173118, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.57)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: right, reward: 2.43064314209
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.4306431420855485, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.43)
65% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 186
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (1, 5), deadline = 25
Simulating trial. . .
epsilon = 0.1557; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1557; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: None, reward: 1.49424344563
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.4942434456321565, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.49)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 2.47688706639
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.47688706639245, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 2.48)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.03688990695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.0368899069548334, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.04)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.65058279248
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.650582792480276, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.65)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.45877848577
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.458778485772042, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.46)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.66408634723
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.6640863472310894, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.66)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: left, reward: 0.910330044525
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 0.9103300445246755, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.91)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 1.76123898467
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.7612389846706102, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.76)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 2.81536944409
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.8153694440918904, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.82)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: 0.986633180293
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.9866331802927615, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.99)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: 2.40857204156
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 2.40857204155621, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.41)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: left, reward: 1.14123500636
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.141235006357822, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.14)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.165267075
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.165267074996091, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.17)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: forward, reward: 1.26501738674
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 1.265017386736547, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.27)
44% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 187
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (2, 2), deadline = 25
Simulating trial. . .
epsilon = 0.1541; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 1.83616158532
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.8361615853151978, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.84)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: left, reward: -9.50745605316
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -9.507456053157307, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.51)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 0.494716958597
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'forward'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 0.49471695859662734, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'forward')
Agent drove right instead of left. (rewarded 0.49)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.99796624884
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.997966248844321, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.00)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: -0.0442880798534
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': -0.04428807985341765, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded -0.04)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 1.44820167881
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.4482016788112482, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 1.45)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 2.66178510138
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.6617851013788103, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.66)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 0.965271105178
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 0.9652711051777014, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.97)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.3304492277
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.3304492277032303, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.33)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: 1.59609998718
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.5960999871834995, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.60)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 0.454578244736
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.45457824473600994, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.45)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: left, reward: 2.72240693458
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 2.7224069345849005, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.72)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: left, reward: 1.07559530984
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 13, 't': 12, 'action': 'left', 'reward': 1.0755953098422661, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.08)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 2.68088248572
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 12, 't': 13, 'action': None, 'reward': 2.6808824857211935, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.68)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.06496411454
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.0649641145385649, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.06)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 2.46904025541
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.469040255406668, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.47)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.23062732146
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.2306273214610555, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.23)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: forward, reward: 1.38409726313
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 1.3840972631323991, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.38)
28% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 188
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (1, 3), deadline = 25
Simulating trial. . .
epsilon = 0.1526; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1526; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1526; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: -39.7691360862
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', 'forward'), 'deadline': 25, 't': 0, 'action': 'left', 'reward': -39.76913608623646, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.77)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.473086498
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.4730864980040705, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.47)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.29753799606
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.297537996064093, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.30)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.29960624526
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.2996062452564017, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.30)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.92344434516
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.923444345156175, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.92)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: 0.520406145228
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 0.5204061452281664, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.52)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: left, reward: -9.49869138836
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': -9.498691388364897, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.50)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 0.674895666753
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 0.6748956667525794, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove forward instead of left. (rewarded 0.67)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 1.46837180327
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.4683718032749706, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.47)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: 1.84312191234
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.8431219123447373, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.84)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 1.60857455472
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 1.6085745547194363, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.61)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 0.904193027597
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': 0.9041930275972689, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.90)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.76184237477
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.7618423747708813, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.76)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 2.6922943681
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 2.6922943680967983, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.69)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 1.06043063579
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.060430635791718, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.06)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: left, reward: 1.48577039287
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 1.485770392871097, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 1.49)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 1.54144803303
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.5414480330294023, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.54)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: -0.286669799064
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': -0.28666979906441936, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent drove right instead of forward. (rewarded -0.29)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 2.10917120597
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 2.109171205973272, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.11)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: left, reward: 2.22405155721
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 2.224051557206173, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.22)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: 1.26702227568
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 5, 't': 20, 'action': 'left', 'reward': 1.2670222756831047, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.27)
16% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 189
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (5, 5), deadline = 20
Simulating trial. . .
epsilon = 0.1511; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1511; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: left, reward: 2.66743913859
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.6674391385931506, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 2.67)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: 0.303181301117
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.30318130111728825, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 0.30)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: forward, reward: 1.0699126667
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.069912666702055, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.07)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 1.71410212757
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.7141021275656736, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.71)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: forward, reward: 1.76132707106
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.7613270710612028, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.76)
75% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 190
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (7, 2), deadline = 25
Simulating trial. . .
epsilon = 0.1496; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 0.54742447307
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 0.5474244730699334, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', 'forward')
Agent drove right instead of left. (rewarded 0.55)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: left, reward: 1.26933548511
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 1.2693354851109022, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.27)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: forward, reward: 1.42017977575
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 1.4201797757531052, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 1.42)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 1.22444230296
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.224442302962329, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.22)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: left, reward: -9.45167035808
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': -9.451670358083316, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -9.45)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: left, reward: 2.30461545274
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 2.304615452737077, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.30)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 0.923666929875
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 0.9236669298747047, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.92)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 0.207800034409
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.20780003440948902, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.21)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 0.957895472583
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 0.9578954725825062, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.96)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: forward, reward: 0.239222541151
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 0.23922254115104646, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 0.24)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: 0.408640007224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.4086400072238242, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 0.41)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.15426027835
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.1542602783470561, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.15)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: -0.0670013272911
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': -0.0670013272911476, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent drove forward instead of right. (rewarded -0.07)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: -5.20141410416
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 12, 't': 13, 'action': None, 'reward': -5.201414104155207, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.20)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: 2.64371866571
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'forward'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 2.6437186657088043, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 2.64)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 2.38024840885
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.3802484088502824, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.38)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 1.54115618927
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.5411561892727186, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.54)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: forward, reward: 2.45830191714
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 2.458301917141808, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.46)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 0.994563178634
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 0.9945631786335343, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.99)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: 1.67485174661
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 1.6748517466062176, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.67)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 0.906357221935
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.9063572219346729, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.91)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: -10.6924716556
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': -10.69247165556154, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.69)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 0.550501322761
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': 0.550501322761267, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.55)
8% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 191
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (5, 2), deadline = 30
Simulating trial. . .
epsilon = 0.1481; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 2.23285550081
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 2.2328555008122652, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.23)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: left, reward: -10.6536526836
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 29, 't': 1, 'action': 'left', 'reward': -10.653652683594945, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -10.65)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 2.91028313035
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 2.910283130348618, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.91)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.82450571556
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.8245057155631719, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.82)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 1.38436187452
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 1.3843618745170128, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.38)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: 2.00744963727
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 2.00744963727049, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.01)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 2.58421475595
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 2.584214755948228, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.58)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: forward, reward: 1.09902418759
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 1.0990241875914597, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 1.10)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: right, reward: 1.60068409525
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 1.6006840952539494, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 1.60)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: right, reward: 2.05956979675
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 2.059569796749706, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 2.06)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: None, reward: 2.64354888828
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.643548888275104, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.64)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: None, reward: -4.68642082284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'right', None), 'deadline': 19, 't': 11, 'action': None, 'reward': -4.686420822840837, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.69)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: right, reward: 2.13655556512
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 2.1365555651245236, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.14)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: left, reward: 0.610309770631
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 0.6103097706308993, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove left instead of right. (rewarded 0.61)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 1.01060723052
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.0106072305199123, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 1.01)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: 2.73529558386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 2.735295583858218, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.74)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.25533096106
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'right'), 'deadline': 14, 't': 16, 'action': None, 'reward': 2.25533096105791, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 2.26)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: 1.67827738905
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 13, 't': 17, 'action': 'left', 'reward': 1.678277389045304, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.68)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 1.75819161704
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 1.7581916170383929, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.76)
37% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 192
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (5, 5), deadline = 25
Simulating trial. . .
epsilon = 0.1466; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.20646161411
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.2064616141127469, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.21)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.13833296709
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.138332967092272, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.14)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 2.85469781138
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.8546978113797508, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.85)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: right, reward: 0.0405826576825
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 0.0405826576824706, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 0.04)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 1.12780271193
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.1278027119333423, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.13)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 1.62064674713
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.6206467471328692, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.62)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.71723443971
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.7172344397087156, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 1.72)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: 2.13078305682
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 2.1307830568223167, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.13)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: left, reward: 1.58964250014
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 1.5896425001399423, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove left instead of forward. (rewarded 1.59)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 1.88503035907
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.8850303590656594, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.89)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 1.68490329756
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.6849032975592404, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.68)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 0.929150044885
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 0.9291500448854833, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 0.93)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 0.863534911723
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 0.8635349117225326, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.86)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: left, reward: 1.93556765972
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 1.9355676597184892, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.94)
44% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 193
\-------------------------
Environment.reset(): Trial set up with start = (2, 6), destination = (6, 5), deadline = 25
Simulating trial. . .
epsilon = 0.1451; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1451; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1451; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1451; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 1.25511479333
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.255114793331072, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.26)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 2.52650066335
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.526500663349797, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.53)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: left, reward: 0.278584047124
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 0.2785840471241696, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.28)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 2.13010120651
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 2.1301012065136966, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.13)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: -4.64263879872
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': -4.642638798717423, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.64)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: left, reward: 0.863811164288
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 0.8638111642880174, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.86)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 2.03265776594
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.032657765937999, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.03)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: left, reward: 0.339826717289
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 0.33982671728931524, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.34)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 1.30358820905
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'forward'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.3035882090527875, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'forward')
Agent followed the waypoint right. (rewarded 1.30)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 2.46054057238
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.4605405723781617, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.46)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: left, reward: 0.9540788368
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 0.9540788368003268, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.95)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: forward, reward: 2.10298473799
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 2.1029847379926627, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.10)
52% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 194
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (5, 5), deadline = 20
Simulating trial. . .
epsilon = 0.1437; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1437; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1437; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: 0.705472786279
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.7054727862789245, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent drove right instead of left. (rewarded 0.71)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: left, reward: 2.12681321628
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 2.1268132162831614, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.13)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: 1.98954706709
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.9895470670882995, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 1.99)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.01419790185
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.0141979018479144, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.01)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 1.62967301663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.6296730166338436, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.63)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: left, reward: 1.07275502552
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.0727550255181582, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 1.07)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.5290199814
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'right'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.5290199814016248, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 2.53)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.03492834728
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.0349283472763813, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.03)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.33882327406
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.3388232740587847, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.34)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: right, reward: 1.8032068667
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.8032068666992105, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.80)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 195
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (4, 4), deadline = 20
Simulating trial. . .
epsilon = 0.1423; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 1.23027053136
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.230270531360855, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.23)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 2.46003711476
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.460037114756592, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 2.46)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.26066847074
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.2606684707441729, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.26)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 2.30655474952
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.3065547495240653, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.31)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 0.849268189017
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.8492681890166697, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.85)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: left, reward: 1.47754310826
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.4775431082623975, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.48)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 2.03447154097
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.034471540970684, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.03)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: 1.43741210759
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.437412107594057, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 1.44)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 196
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (5, 5), deadline = 20
Simulating trial. . .
epsilon = 0.1409; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1409; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: left, reward: 2.22717355448
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.2271735544776528, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.23)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.63636192946
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.6363619294577436, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.64)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.32516286092
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.3251628609202024, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.33)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 2.21176389685
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.211763896854434, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.21)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.22098633859
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.2209863385854818, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.22)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: forward, reward: 0.188070995452
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 0.1880709954519204, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'forward')
Agent drove forward instead of left. (rewarded 0.19)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 1.05548708177
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.0554870817675175, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.06)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 2.0499495776
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.049949577595355, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.05)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 1.08755725515
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.0875572551493, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.09)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 1.68138194884
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.6813819488357875, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.68)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: forward, reward: 0.650498441921
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 0.6504984419205223, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 0.65)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 0.868059859221
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 0.868059859220544, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.87)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.3646458463
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.3646458462977773, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.36)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: left, reward: 1.37078602431
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.3707860243109307, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.37)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: 1.23295990682
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 1.2329599068230328, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.23)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: forward, reward: 1.9796128861
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 1.979612886099807, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.98)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 0.574448439897
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.574448439897095, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.57)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 0.388580159434
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.3885801594340963, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.39)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 2.07017875887
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 2.070178758866107, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.07)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: 0.0100903991322
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'forward'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 0.010090399132209993, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, 'right', 'forward')
Agent drove forward instead of left. (rewarded 0.01)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 197
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (6, 6), deadline = 20
Simulating trial. . .
epsilon = 0.1395; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: left, reward: 1.24311980388
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.2431198038800255, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.24)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 2.46848634827
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.4684863482730055, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.47)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 2.45528103967
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.455281039671422, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.46)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: left, reward: -9.34019678165
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -9.340196781650615, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.34)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: forward, reward: 1.72410637797
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.7241063779691879, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.72)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.14624278947
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.1462427894663687, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.15)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.05206941101
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.0520694110089341, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.05)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: left, reward: 1.53663997145
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.536639971452978, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.54)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 1.4133643113
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.4133643112977337, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.41)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: left, reward: 0.991046335669
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 0.9910463356693637, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', None)
Agent drove left instead of forward. (rewarded 0.99)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 2.72432733038
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'forward'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 2.7243273303810573, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 2.72)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 198
\-------------------------
Environment.reset(): Trial set up with start = (7, 4), destination = (1, 6), deadline = 20
Simulating trial. . .
epsilon = 0.1381; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 1.68931120916
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.6893112091621254, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.69)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 2.91887870252
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.918878702523741, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.92)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 1.28041944176
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.2804194417624049, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.28)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 2.52436109285
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.5243610928521587, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.52)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 2.74677739687
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.746777396871599, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.75)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 2.32729791833
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.3272979183318023, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.33)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: 1.67447659591
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'right', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.6744765959076033, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'right', None)
Agent drove forward instead of left. (rewarded 1.67)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 1.07091351738
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.0709135173807351, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.07)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: -10.486753238
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -10.486753238001125, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -10.49)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 1.46357149424
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.463571494239709, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.46)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: left, reward: 1.3990484915
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.3990484914979044, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.40)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: left, reward: 2.07311043389
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 2.073110433886562, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.07)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 1.73665041682
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.7366504168186427, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.74)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 1.02460455546
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.0246045554586065, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.02)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 1.37683600557
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.3768360055676254, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 1.38)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 2.00766118507
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.0076611850658663, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.01)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 1.32545158224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.3254515822411417, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.33)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: 1.35803833511
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 1.358038335107478, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.36)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 1.54299724665
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 1.5429972466492834, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.54)
5% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 199
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (8, 3), deadline = 30
Simulating trial. . .
epsilon = 0.1367; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: left, reward: 0.549161495923
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 0.5491614959233757, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.55)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: None, reward: 2.00430382854
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.0043038285431516, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.00)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: left, reward: -10.6819821548
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 28, 't': 2, 'action': 'left', 'reward': -10.681982154838941, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent attempted driving left through a red light. (rewarded -10.68)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: right, reward: 1.31315606925
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 1.3131560692511277, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.31)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.19764024054
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.197640240542566, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.20)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.47424845732
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.4742484573195402, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.47)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: forward, reward: 1.39993136177
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 1.39993136177156, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.40)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: 1.71211735398
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 1.7121173539848542, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.71)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 2.59632030968
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.5963203096826324, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.60)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.89607077668
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.8960707766779428, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.90)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 0.383731080779
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 0.38373108077850837, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.38)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: left, reward: 2.71844455662
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 2.7184445566226456, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.72)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 0.734417601616
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', 'right'), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': 0.7344176016161211, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', 'right')
Agent drove forward instead of right. (rewarded 0.73)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 1.46068320938
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 1.460683209377689, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.46)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 1.15568170405
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.1556817040522842, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 1.16)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 2.68271066212
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'right', None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 2.6827106621244257, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', None)
Agent followed the waypoint right. (rewarded 2.68)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: left, reward: 0.753450708092
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 0.7534507080923722, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 0.75)
43% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 200
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (5, 6), deadline = 25
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 1.35561386286
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.355613862855714, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.36)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 2.11050958196
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.110509581959145, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.11)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 2.7578722751
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.7578722750957527, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.76)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: 1.64727115241
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.6472711524129948, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.65)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 2.02913089905
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.0291308990455335, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.03)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: forward, reward: 1.03501548282
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.0350154828248348, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.04)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: right, reward: 2.11562541761
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.1156254176052403, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.12)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: left, reward: 0.96259724092
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 0.9625972409204213, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.96)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: right, reward: 1.15377835087
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'forward'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.1537783508699226, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'forward')
Agent followed the waypoint right. (rewarded 1.15)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: right, reward: 0.870589020491
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.8705890204912135, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.87)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 201
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (3, 7), deadline = 20
Simulating trial. . .
epsilon = 0.1340; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1340; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1340; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1340; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: 2.79961489416
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.7996148941580503, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.80)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 1.38361294367
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.3836129436725897, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.38)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 2.31728126462
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.3172812646171024, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.32)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: left, reward: 2.19789854681
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 2.1978985468065035, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.20)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: None, reward: 1.14648108418
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.146481084175639, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.15)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 1.22127682176
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.2212768217583982, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.22)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: left, reward: 2.19888156729
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.1988815672916395, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 2.20)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 1.66307217116
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.6630721711626377, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.66)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: left, reward: -39.769111309
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -39.76911130897286, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.77)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 2.11026496589
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.11026496589011, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.11)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: left, reward: 0.803534072873
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 0.8035340728733513, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.80)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 2.0681725055
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.0681725054977416, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.07)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 1.1101498502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.110149850203214, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.11)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: right, reward: 1.7673898169
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.7673898169001374, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.77)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 202
\-------------------------
Environment.reset(): Trial set up with start = (2, 6), destination = (7, 7), deadline = 20
Simulating trial. . .
epsilon = 0.1327; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: left, reward: 1.64080615383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', 'left'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.6408061538252083, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', 'left')
Agent drove left instead of forward. (rewarded 1.64)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: forward, reward: 0.774872639455
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 0.7748726394550683, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 0.77)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 1.9502535397
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.9502535397040235, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.95)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 1.09477559222
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.0947755922229943, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.09)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 1.96264698785
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.962646987848045, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.96)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: 0.980726312984
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.980726312983637, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.98)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: left, reward: 1.83902317193
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.8390231719331858, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.84)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 2.77203716168
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 2.7720371616780266, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.77)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 2.67512263602
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 2.6751226360211846, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 2.68)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 0.862724557606
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.8627245576064528, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 0.86)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 1.29711865678
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.297118656782004, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.30)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: -9.51213467044
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', 'left', 'right'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': -9.512134670444764, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', 'right')
Agent attempted driving forward through a red light. (rewarded -9.51)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 1.32851574481
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.3285157448092728, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.33)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 0.802208169461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.802208169461195, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', 'left')
Agent drove right instead of forward. (rewarded 0.80)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: -39.4173759484
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -39.417375948449866, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.42)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.63997097211
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.6399709721071423, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.64)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 1.11487660598
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.1148766059796806, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.11)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: -0.189694871251
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -0.18969487125134255, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent drove forward instead of left. (rewarded -0.19)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: -0.692331441865
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'forward'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': -0.6923314418646067, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'forward')
Agent drove right instead of left. (rewarded -0.69)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 0.619204832673
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.6192048326733666, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 0.62)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 203
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (1, 3), deadline = 25
Simulating trial. . .
epsilon = 0.1313; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1313; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1313; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 0.515075533027
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 0.5150755330267944, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.52)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: 2.75662167454
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'forward'), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 2.7566216745355185, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 2.76)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: 1.84420470747
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 1.8442047074736567, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.84)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.61114780843
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.6111478084293758, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.61)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.32531017542
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.3253101754186687, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.33)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 2.67892264997
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.6789226499680105, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.68)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 1.77853523828
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.7785352382812645, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.78)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 1.27972341677
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.279723416766995, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 1.28)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 0.922196586345
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 17, 't': 8, 'action': None, 'reward': 0.9221965863445942, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.92)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 1.63675810847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.6367581084665064, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.64)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: left, reward: 1.42097448166
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 1.4209744816592302, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.42)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: 1.74203271837
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.7420327183682685, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent drove right instead of left. (rewarded 1.74)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 1.73819554941
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 1.7381955494090007, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent drove forward instead of right. (rewarded 1.74)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 2.19624518003
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 2.1962451800255707, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.20)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: left, reward: -20.886716814
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', 'forward'), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -20.886716814033605, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.89)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 1.9892090742
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.989209074204335, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.99)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 1.0916791881
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.091679188097902, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.09)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 2.49968486151
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 2.4996848615098526, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 2.50)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 0.864351136158
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': 0.8643511361583871, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 0.86)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 2.4553587035
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'right'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 2.455358703504688, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'right')
Agent followed the waypoint right. (rewarded 2.46)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 204
\-------------------------
Environment.reset(): Trial set up with start = (4, 7), destination = (6, 5), deadline = 20
Simulating trial. . .
epsilon = 0.1300; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 2.61546761683
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.6154676168309106, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.62)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 1.34095150364
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.3409515036439967, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.34)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: 1.86861458125
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.868614581245922, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.87)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 1.01935820635
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.0193582063530937, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', None)
Agent drove forward instead of left. (rewarded 1.02)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: left, reward: 2.91946311865
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 2.9194631186452726, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.92)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: forward, reward: -9.97216019776
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': -9.972160197757049, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.97)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: left, reward: 2.21727435859
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.2172743585942647, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.22)
65% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 205
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (2, 6), deadline = 25
Simulating trial. . .
epsilon = 0.1287; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1287; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 1.34028637551
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.340286375505773, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 1.34)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: None, reward: 1.51909850725
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.5190985072492822, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 1.52)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: None, reward: 2.7135612651
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.7135612651012773, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.71)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 2.47609680856
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 2.476096808556239, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.48)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 0.104470329295
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 0.10447032929487399, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 0.10)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: 1.21149234314
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.2114923431409244, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.21)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: -4.68711981072
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': -4.687119810724379, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.69)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: 0.952415220826
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 0.9524152208255765, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 0.95)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: 1.61063865361
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 1.6106386536054456, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.61)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 2.44961454219
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.4496145421894964, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.45)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: -10.8748203022
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': -10.874820302234214, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.87)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 2.45596623927
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 2.455966239273546, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 2.46)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 0.125090377515
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'right'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 0.125090377514675, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'right')
Agent drove right instead of left. (rewarded 0.13)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 1.05094818026
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.0509481802560348, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.05)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: 1.69367593242
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'right'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 1.6936759324244406, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 1.69)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 2.05752053507
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 2.0575205350745356, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.06)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 0.866873900484
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 0.8668739004840735, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.87)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: 1.06966920618
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 1.0696692061794948, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 1.07)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 1.98435048971
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 1.98435048970867, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.98)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 1.66510405739
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 1.6651040573859992, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.67)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.79695730255
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.796957302546594, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.80)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.14910874501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 4, 't': 21, 'action': None, 'reward': 2.149108745007327, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.15)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 2.02184497565
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': 2.0218449756530914, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.02)
8% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 206
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (1, 7), deadline = 30
Simulating trial. . .
epsilon = 0.1275; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 2.49507702546
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.4950770254629653, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.50)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 1.71325856565
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.71325856565142, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.71)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 1.42694263097
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.4269426309690147, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.43)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 2.96651446793
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.966514467932675, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.97)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: forward, reward: 0.557677042788
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 0.5576770427876822, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.56)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: left, reward: 2.46142725238
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 2.4614272523768106, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.46)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 2.19224381741
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 2.1922438174126886, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.19)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: left, reward: 0.655263245735
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 0.6552632457349564, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', None)
Agent drove left instead of forward. (rewarded 0.66)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: None, reward: 0.966049162484
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 0.9660491624835184, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.97)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: forward, reward: 0.274131421145
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 0.27413142114505007, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent drove forward instead of right. (rewarded 0.27)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 0.918973191254
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 0.9189731912544004, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 0.92)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: -9.84612346939
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': -9.846123469385656, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.85)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.79432773999
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.7943277399889141, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.79)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: 0.892107804452
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'right'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 0.8921078044523054, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'right')
Agent drove right instead of left. (rewarded 0.89)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 1.52245460083
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.5224546008299462, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.52)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 1.98146293888
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.9814629388818263, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.98)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 1.73375421894
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 14, 't': 16, 'action': None, 'reward': 1.73375421894106, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.73)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 1.62620819925
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'forward'), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 1.6262081992531074, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.63)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.14521593301
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'right'), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.1452159330082547, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'right')
Agent properly idled at a red light. (rewarded 1.15)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: 1.772814305
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 19, 'action': 'left', 'reward': 1.7728143049979952, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.77)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: forward, reward: 1.64249744968
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 20, 'action': 'forward', 'reward': 1.6424974496837377, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.64)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 207
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (7, 6), deadline = 25
Simulating trial. . .
epsilon = 0.1262; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1262; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 0.240316976743
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 0.24031697674334906, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove right instead of left. (rewarded 0.24)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 2.05940290296
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.059402902959544, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.06)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: forward, reward: 0.959300342016
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 0.9593003420161303, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.96)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: right, reward: 1.81110215359
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.811102153585822, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 1.81)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: forward, reward: -9.00509123405
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -9.005091234048262, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.01)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 2.68084694059
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.6808469405948028, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.68)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 1.67191681461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.6719168146057124, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.67)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: 2.61939187987
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 2.619391879868167, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 2.62)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.62727529103
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.6272752910325807, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.63)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 1.77745136603
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.7774513660317677, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 1.78)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: -10.2468058365
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': -10.246805836479842, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.25)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: 1.5709084888
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.5709084888017022, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.57)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.86755123422
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.867551234224449, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.87)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: left, reward: 2.02478722019
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 2.024787220193809, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.02)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 2.079614208
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.079614207995956, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.08)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 1.27437455296
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.2743745529564694, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.27)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: forward, reward: 1.00004218613
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 1.0000421861337052, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.00)
32% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 208
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (7, 7), deadline = 25
Simulating trial. . .
epsilon = 0.1249; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1249; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 1.00463872742
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.0046387274206026, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 1.00)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: left, reward: 2.74150266298
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 2.7415026629790367, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.74)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: 1.61679191838
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 1.616791918384236, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.62)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 2.82215993908
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.8221599390787455, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.82)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 1.88512822893
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 1.8851282289301374, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.89)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 2.3372399911
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 2.337239991098274, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.34)
76% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 209
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (5, 7), deadline = 20
Simulating trial. . .
epsilon = 0.1237; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1237; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1237; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 1.55348094865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.5534809486515497, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.55)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: 1.51632857718
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.5163285771808894, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.52)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 0.752864641429
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.7528646414288053, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.75)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: -40.9017016332
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -40.901701633219716, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.90)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: left, reward: 1.7107089733
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.7107089732958234, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.71)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: left, reward: 2.31470123305
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.3147012330467485, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.31)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 1.9680251465
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.9680251464983185, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.97)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: 0.210110452898
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 0.21011045289807595, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove forward instead of right. (rewarded 0.21)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 1.29351333924
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.2935133392413953, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.29)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 2.60571972128
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.605719721275682, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.61)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 2.60317697296
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.603176972960868, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.60)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: right, reward: 2.65017896611
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'right'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 2.6501789661058526, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'right')
Agent followed the waypoint right. (rewarded 2.65)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 210
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (4, 2), deadline = 25
Simulating trial. . .
epsilon = 0.1225; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.4774899467
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.4774899466993703, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.48)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 2.66154965833
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.6615496583339677, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 2.66)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 1.04347154973
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.043471549727192, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.04)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.97165958198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.9716595819812872, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.97)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 0.957584974601
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 0.9575849746011083, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.96)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 1.93484404777
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.934844047772439, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.93)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: left, reward: 0.828874964521
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 0.8288749645210906, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.83)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: right, reward: 2.32514415119
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 2.32514415119116, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 2.33)
68% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 211
\-------------------------
Environment.reset(): Trial set up with start = (7, 6), destination = (3, 5), deadline = 25
Simulating trial. . .
epsilon = 0.1212; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1212; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: forward, reward: 0.652127460534
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'left', 'forward'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 0.6521274605340224, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'left', 'forward')
Agent drove forward instead of right. (rewarded 0.65)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: forward, reward: -10.056899281
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': -10.056899281016902, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.06)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.88185693788
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.881856937879038, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.88)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 2.77477254139
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.774772541390787, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.77)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 2.0877884149
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.0877884148963437, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.09)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 0.407900513711
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.40790051371082947, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.41)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 1.83571331549
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.8357133154868266, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.84)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: left, reward: 2.11330241327
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 2.1133024132668488, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 2.11)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: -0.118076877604
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': -0.11807687760380603, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove right instead of forward. (rewarded -0.12)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.6819019025
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.6819019025021178, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.68)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 0.878098805989
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 0.8780988059889347, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 0.88)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 2.37540878754
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.375408787541768, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.38)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 0.818500127506
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 0.8185001275064274, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', 'left')
Agent properly idled at a red light. (rewarded 0.82)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: left, reward: 2.02958831075
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 2.029588310747888, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.03)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: left, reward: 1.7655594577
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 1.765559457704862, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 1.77)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: 1.66639636726
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 1.6663963672596522, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.67)
36% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 212
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (7, 2), deadline = 25
Simulating trial. . .
epsilon = 0.1200; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 1.97619660482
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.9761966048195767, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 1.98)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.08086429777
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.0808642977690686, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', None)
Agent properly idled at a red light. (rewarded 1.08)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.2308886373
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.2308886373037775, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.23)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 2.61625535286
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.6162553528646817, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.62)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 2.08591196982
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.085911969817616, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.09)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 2.51325984428
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.5132598442835468, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.51)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 1.74274877716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.7427487771569083, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.74)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: -4.1317028632
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 18, 't': 7, 'action': None, 'reward': -4.131702863196713, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.13)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 1.97524731636
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 1.975247316363501, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.98)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 2.84997971559
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.849979715592455, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.85)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.79095327867
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.7909532786723927, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.79)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 0.574532692368
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 0.5745326923684381, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.57)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 0.854650322516
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 0.8546503225156385, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.85)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: 0.756443147963
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 0.7564431479632674, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 0.76)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: forward, reward: 1.07810211969
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'left'), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 1.0781021196937406, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'left')
Agent drove forward instead of left. (rewarded 1.08)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: 1.19611384074
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 1.196113840741363, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.20)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: left, reward: 0.639053453808
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 0.6390534538076817, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 0.64)
32% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 213
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (1, 4), deadline = 20
Simulating trial. . .
epsilon = 0.1188; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1188; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1188; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: -39.5329755628
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -39.53297556282886, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.53)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.65132316404
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.6513231640375166, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.65)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: 1.5196975851
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.5196975851000518, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.52)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: 1.6013703677
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 1.601370367701357, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.60)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.86429714625
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.8642971462534157, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.86)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 1.38554257254
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.3855425725418016, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.39)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 1.11789379265
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.1178937926461225, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.12)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: left, reward: 0.775795066848
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 0.7757950668477263, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', None)
Agent drove left instead of right. (rewarded 0.78)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: forward, reward: 1.6498543003
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.6498543003010684, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.65)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 1.7162659466
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.7162659465969168, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 1.72)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 2.74336599769
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.7433659976928126, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.74)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 0.770061393345
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.7700613933452984, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 0.77)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.10831428083
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.1083142808303883, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.11)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 1.33762012796
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.337620127959943, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.34)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.40374836529
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.4037483652913956, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.40)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 0.543255207589
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.5432552075891275, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 0.54)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 1.34479920472
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.3447992047162782, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.34)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.22379105151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 2.223791051510305, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.22)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.1470398401
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 2.1470398400958675, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.15)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.39352613013
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.3935261301312714, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.39)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 214
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (5, 2), deadline = 20
Simulating trial. . .
epsilon = 0.1177; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1177; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1177; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: 1.04318806513
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.0431880651255265, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.04)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 1.29501922144
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.2950192214361147, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.30)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: forward, reward: 1.47082216118
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.470822161175593, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 1.47)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 0.856920543254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 0.8569205432537167, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 0.86)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 0.172856156155
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 0.17285615615496897, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.17)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 0.54138001598
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 0.5413800159801867, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.54)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: -40.3944327015
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -40.39443270145911, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.39)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 0.988736961297
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 0.9887369612973527, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.99)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 2.76017605915
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.7601760591478466, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.76)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: left, reward: -19.3860824446
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', 'forward'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': -19.38608244464513, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.39)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: left, reward: 1.06120153489
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.0612015348880086, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', None)
Agent drove left instead of forward. (rewarded 1.06)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 1.59734930286
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.5973493028576007, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.60)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 1.92272496339
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 1.922724963389498, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.92)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: forward, reward: 2.34641620782
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 2.3464162078208344, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.35)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: right, reward: 1.69099528136
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.690995281357498, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 1.69)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: forward, reward: 2.46226807285
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 2.4622680728475235, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.46)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: right, reward: 0.820723883529
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.8207238835293971, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.82)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 1.00960230399
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 1.0096023039901536, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove right instead of left. (rewarded 1.01)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 1.67254836374
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.6725483637384193, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.67)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 1.00167092435
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 1.001670924353436, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.00)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 215
\-------------------------
Environment.reset(): Trial set up with start = (5, 5), destination = (8, 2), deadline = 30
Simulating trial. . .
epsilon = 0.1165; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1165; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: right, reward: 1.03921459255
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.039214592554942, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.04)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: 1.8066733152
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'right', None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.806673315201658, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'right', None)
Agent drove right instead of left. (rewarded 1.81)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: right, reward: 1.08020867984
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 1.080208679838674, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.08)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 1.14462886105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.144628861053798, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.14)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: 1.70564757621
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'right', None), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 1.7056475762074914, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'right', None)
Agent drove forward instead of left. (rewarded 1.71)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: forward, reward: 1.38432392831
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.3843239283112871, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', None)
Agent drove forward instead of left. (rewarded 1.38)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: left, reward: 1.74870768406
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 1.7487076840573152, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.75)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: -10.1440874805
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': -10.144087480489253, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.14)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 1.64542236623
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.6454223662258327, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.65)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 2.21727120658
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 21, 't': 9, 'action': None, 'reward': 2.2172712065790723, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.22)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: 1.6661049483
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 1.6661049482972003, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.67)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.32670420215
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.3267042021464097, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.33)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 2.34878867681
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.3487886768050688, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.35)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: -0.129497938899
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'right'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': -0.12949793889889727, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'right')
Agent drove right instead of forward. (rewarded -0.13)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: 1.57055796207
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': 1.5705579620650252, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove forward instead of left. (rewarded 1.57)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.03617926886
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.0361792688614342, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.04)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.83929715148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 14, 't': 16, 'action': None, 'reward': 1.8392971514805554, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.84)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: forward, reward: -0.134870299564
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': -0.13487029956420926, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded -0.13)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: 1.44824133758
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 1.448241337579628, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent drove right instead of left. (rewarded 1.45)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.50492730413
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 1.5049273041337714, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.50)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 0.794855965015
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 0.7948559650150195, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 0.79)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: left, reward: 0.399495967209
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 9, 't': 21, 'action': 'left', 'reward': 0.39949596720946057, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove left instead of forward. (rewarded 0.40)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: left, reward: 0.814279404157
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 22, 'action': 'left', 'reward': 0.8142794041571852, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 0.81)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 1.53238745261
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'right'), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 1.5323874526099572, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'right')
Agent followed the waypoint right. (rewarded 1.53)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 0.625046796785
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': None, 'reward': 0.6250467967846294, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.63)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 1.48964883787
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 1.489648837870193, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.49)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 0.930114148331
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 4, 't': 26, 'action': None, 'reward': 0.9301141483305249, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.93)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.00088676057
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 3, 't': 27, 'action': None, 'reward': 1.0008867605686926, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.00)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.48153298816
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 2, 't': 28, 'action': None, 'reward': 1.4815329881611174, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.48)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 0.409765253052
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 1, 't': 29, 'action': 'forward', 'reward': 0.40976525305244116, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 0.41)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 216
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (5, 4), deadline = 25
Simulating trial. . .
epsilon = 0.1153; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: left, reward: 2.63477120682
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 2.6347712068177036, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.63)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.78254460843
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.7825446084271561, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.78)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 2.63196021847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.63196021847038, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.63)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.56572709216
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.5657270921600994, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.57)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.81469940993
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.8146994099250358, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.81)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.79970358458
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.7997035845775196, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.80)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 1.41404847012
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.4140484701150988, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.41)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: left, reward: 0.26997024186
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 0.2699702418597729, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.27)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 2.23702016184
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 2.2370201618381476, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.24)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: forward, reward: 1.40691403081
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'right'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.4069140308068278, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'right')
Agent followed the waypoint forward. (rewarded 1.41)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: right, reward: 2.48624277011
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 2.486242770112705, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.49)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: left, reward: -10.8237126576
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': -10.82371265760342, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.82)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: None, reward: 1.99067594182
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.9906759418186075, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.99)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: forward, reward: 2.3183624237
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 2.318362423704845, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.32)
44% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 217
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (6, 5), deadline = 25
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 1.02540386263
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.0254038626349211, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.03)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: left, reward: 0.189432208883
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 0.1894322088832907, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', None)
Agent drove left instead of right. (rewarded 0.19)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: forward, reward: 1.30022117169
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 1.3002211716933803, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent followed the waypoint forward. (rewarded 1.30)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 1.34109778435
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.3410977843510514, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.34)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: left, reward: 0.809990956845
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 0.8099909568450817, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.81)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 2.40646129387
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 2.4064612938680487, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.41)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 0.583386472888
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'right'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.5833864728882338, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'right')
Agent drove right instead of forward. (rewarded 0.58)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 0.136042566189
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'right'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.13604256618892463, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'right')
Agent drove right instead of left. (rewarded 0.14)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: left, reward: 1.37036993861
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 1.3703699386052242, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.37)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: 2.46619160979
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 2.466191609794957, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.47)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 1.51606082165
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 1.516060821652643, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.52)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 1.56118175065
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.5611817506512657, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.56)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.08469903569
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'right'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.0846990356944053, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 2.08)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 2.02783841576
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 2.0278384157580325, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.03)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: None, reward: 2.33294755757
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.332947557565184, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.33)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: forward, reward: 1.31296317805
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 1.312963178050915, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.31)
36% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 218
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (1, 6), deadline = 25
Simulating trial. . .
epsilon = 0.1130; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1130; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1130; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1130; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 2.8662091186
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.8662091185982312, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.87)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 1.3478590529
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.3478590528965568, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.35)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 2.86129476847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.8612947684678214, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.86)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 2.12045784536
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.12045784536019, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.12)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: left, reward: 1.04027146054
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 1.040271460539753, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.04)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: right, reward: 0.166192381265
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.16619238126521385, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent drove right instead of left. (rewarded 0.17)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: -40.2678714995
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', 'right'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -40.2678714994645, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', 'right')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.27)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: left, reward: 1.96644878991
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 1.9664487899112442, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 1.97)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 2.19330518456
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.193305184557305, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.19)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: left, reward: 1.61561579428
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.6156157942848752, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 1.62)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.09491025153
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.0949102515280007, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.09)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 1.68046148559
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.6804614855912405, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.68)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 0.707202332722
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 0.7072023327221615, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.71)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 2.61841907609
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 12, 't': 13, 'action': None, 'reward': 2.6184190760926938, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.62)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 1.39828264875
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.3982826487517408, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.40)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 0.67108808427
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': None, 'reward': 0.671088084270232, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.67)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 1.47474106821
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.4747410682078685, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.47)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: forward, reward: 0.688058252525
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 0.6880582525250618, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove forward instead of left. (rewarded 0.69)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: None, reward: 1.17058657808
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.1705865780765077, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.17)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: None, reward: 2.32582367578
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'left'), 'deadline': 6, 't': 19, 'action': None, 'reward': 2.3258236757843944, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.33)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: None, reward: 0.949422696161
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.9494226961605032, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.95)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: left, reward: 1.13452389204
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 1.1345238920379237, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.13)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: left, reward: 2.00277861366
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 3, 't': 22, 'action': 'left', 'reward': 2.002778613664847, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.00)
8% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 219
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (1, 7), deadline = 35
Simulating trial. . .
epsilon = 0.1119; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: forward, reward: 1.39929080777
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'forward'), 'deadline': 35, 't': 0, 'action': 'forward', 'reward': 1.3992908077655728, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.40)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: -20.8572604292
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 34, 't': 1, 'action': 'left', 'reward': -20.857260429212115, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.86)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: left, reward: 0.27274012493
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', None), 'deadline': 33, 't': 2, 'action': 'left', 'reward': 0.2727401249304737, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', None)
Agent drove left instead of forward. (rewarded 0.27)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: right, reward: 1.86268414082
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 32, 't': 3, 'action': 'right', 'reward': 1.8626841408215578, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.86)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: right, reward: 0.552477229839
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 31, 't': 4, 'action': 'right', 'reward': 0.5524772298390594, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 0.55)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: left, reward: -10.5964537254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 30, 't': 5, 'action': 'left', 'reward': -10.596453725370552, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.60)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 0.530031548552
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 29, 't': 6, 'action': 'right', 'reward': 0.5300315485519806, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.53)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: 1.66674414932
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 28, 't': 7, 'action': 'right', 'reward': 1.6667441493226205, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.67)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: right, reward: 1.36016062703
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'left'), 'deadline': 27, 't': 8, 'action': 'right', 'reward': 1.3601606270287363, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'left')
Agent followed the waypoint right. (rewarded 1.36)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 2.54600975534
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 26, 't': 9, 'action': None, 'reward': 2.546009755344155, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.55)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 2.03495581859
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 25, 't': 10, 'action': None, 'reward': 2.034955818585962, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.03)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 0.920827459087
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 24, 't': 11, 'action': None, 'reward': 0.920827459086899, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.92)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 1.81913879441
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 12, 'action': None, 'reward': 1.8191387944116755, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.82)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: 1.42202076683
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 22, 't': 13, 'action': 'forward', 'reward': 1.4220207668333902, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.42)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: right, reward: 0.972750114653
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'right'), 'deadline': 21, 't': 14, 'action': 'right', 'reward': 0.9727501146531982, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'right')
Agent drove right instead of forward. (rewarded 0.97)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: left, reward: 1.93947774711
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 20, 't': 15, 'action': 'left', 'reward': 1.9394777471083775, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.94)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.47883742602
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 19, 't': 16, 'action': None, 'reward': 1.47883742602311, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.48)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.48102327896
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 17, 'action': None, 'reward': 1.4810232789564617, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.48)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.38564591726
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 18, 'action': None, 'reward': 1.3856459172587623, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.39)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 2.51958276129
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 16, 't': 19, 'action': None, 'reward': 2.5195827612938184, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.52)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 0.973357495733
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', None), 'deadline': 15, 't': 20, 'action': None, 'reward': 0.9733574957330755, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', None)
Agent properly idled at a red light. (rewarded 0.97)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: left, reward: 2.06216128771
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 14, 't': 21, 'action': 'left', 'reward': 2.06216128770994, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 2.06)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: -0.280073808218
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 13, 't': 22, 'action': 'right', 'reward': -0.28007380821815264, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded -0.28)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: 1.70097185923
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 23, 'action': 'left', 'reward': 1.7009718592306267, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.70)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: left, reward: 1.49618029149
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 11, 't': 24, 'action': 'left', 'reward': 1.4961802914875608, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.50)
29% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 1.05956196232
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 10, 't': 25, 'action': 'right', 'reward': 1.0595619623177173, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.06)
26% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 220
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (3, 4), deadline = 25
Simulating trial. . .
epsilon = 0.1108; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 2.63139466167
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.631394661674274, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.63)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 2.1891178888
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.1891178888019844, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.19)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 2.72952830232
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.72952830232209, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.73)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 2.17132432682
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.1713243268161326, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.17)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.59828812874
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'right'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 1.5982881287360633, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'right')
Agent followed the waypoint forward. (rewarded 1.60)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: left, reward: -40.111489276
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': -40.111489276006296, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.11)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 2.64055486982
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.6405548698198027, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.64)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: 0.977839984469
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 0.9778399844690528, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 0.98)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: right, reward: 2.1320472693
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 2.1320472692976526, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.13)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 2.34536693542
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.3453669354184745, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.35)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 2.48275964324
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.4827596432362444, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.48)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 2.04452206591
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.0445220659107513, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.04)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 0.777462655484
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 0.7774626554843151, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.78)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: forward, reward: 0.725537834653
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 0.7255378346531891, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.73)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: forward, reward: 0.888765259049
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 0.8887652590488434, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent drove forward instead of right. (rewarded 0.89)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: 2.48504560541
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 2.48504560540672, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.49)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 0.817565520667
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 0.817565520667191, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 0.82)
32% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 221
\-------------------------
Environment.reset(): Trial set up with start = (7, 6), destination = (2, 5), deadline = 20
Simulating trial. . .
epsilon = 0.1097; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 0.640716268888
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.640716268887505, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'left')
Agent drove right instead of left. (rewarded 0.64)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: -4.01487058258
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', 'right'), 'deadline': 19, 't': 1, 'action': None, 'reward': -4.014870582575711, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.01)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: left, reward: 1.14304711237
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.1430471123656374, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent drove left instead of right. (rewarded 1.14)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: left, reward: 1.30796591124
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 1.3079659112391575, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.31)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.65782728317
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.6578272831727265, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.66)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 0.738831921256
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.7388319212564094, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 0.74)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: forward, reward: 0.583025025698
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'forward'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 0.5830250256979639, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'forward')
Agent drove forward instead of left. (rewarded 0.58)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 1.96500670909
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.9650067090867374, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.97)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 1.78463744753
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.7846374475263125, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.78)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: forward, reward: 0.482329835864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.4823298358641841, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 0.48)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: left, reward: 1.6297286317
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.6297286316950899, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.63)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 0.755567241022
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.7555672410220056, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.76)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 0.755295965802
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.7552959658018856, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.76)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.46111739456
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.4611173945612004, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.46)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 2.36867099878
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.368670998775088, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.37)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 2.36627457322
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.3662745732214248, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.37)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: forward, reward: 0.700318283241
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 0.7003182832414037, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.70)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 1.03866120487
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.0386612048670825, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.04)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 0.820516058412
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.8205160584115749, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.82)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: -0.059451466217
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': -0.0594514662169735, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded -0.06)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 222
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (1, 3), deadline = 30
Simulating trial. . .
epsilon = 0.1086; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: right, reward: 1.83040140124
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.8304014012353753, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.83)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: forward, reward: 2.27624663983
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 2.2762466398319994, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.28)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: 1.37802862609
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 1.3780286260874868, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 1.38)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.51682382613
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.5168238261252567, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.52)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: 1.02158845813
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 1.021588458132154, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove forward instead of left. (rewarded 1.02)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.34784718776
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.3478471877582285, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.35)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: left, reward: -10.3573620668
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': -10.35736206680179, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent attempted driving left through a red light. (rewarded -10.36)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 0.708474800776
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 0.7084748007763052, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 0.71)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 1.9525286544
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 1.952528654400834, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.95)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: -0.042769997894
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': -0.0427699978940117, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded -0.04)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 1.82337912471
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.8233791247137399, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.82)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 2.34292367879
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.3429236787931256, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.34)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 1.700097107
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.7000971070013136, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.70)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: forward, reward: 2.04588689532
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 2.045886895324691, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.05)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 2.44201120277
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 2.4420112027670573, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.44)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: forward, reward: 2.04645691385
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': 2.046456913847795, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.05)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: forward, reward: 1.81630485996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': 1.81630485996238, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.82)
43% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 223
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (8, 2), deadline = 20
Simulating trial. . .
epsilon = 0.1075; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: left, reward: 2.12674009685
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.126740096846196, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.13)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: forward, reward: -10.6957504263
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -10.695750426296739, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -10.70)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 1.80069497946
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.800694979462248, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.80)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 1.75635016833
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.7563501683299791, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.76)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: left, reward: 2.2801325092
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 2.2801325092009277, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.28)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 2.80435838777
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.8043583877663454, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.80)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 1.66417431477
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.6641743147718386, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.66)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 2.11769496949
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.117694969494867, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.12)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 2.33912243174
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.339122431735707, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.34)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: forward, reward: 2.13503879881
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 2.1350387988102915, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.14)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: 2.36310418241
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 2.3631041824058556, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.36)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 224
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (1, 7), deadline = 30
Simulating trial. . .
epsilon = 0.1065; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1065; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.26478935788
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.264789357877873, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.26)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 2.80378315327
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.80378315327017, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.80)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.02877989742
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.028779897421349, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.03)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.02707868564
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.0270786856441592, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.03)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 1.99465161807
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 1.9946516180712828, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.99)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 0.476043557304
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'right'), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 0.47604355730405234, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'right')
Agent drove right instead of forward. (rewarded 0.48)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: 2.74073013259
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 2.740730132590462, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.74)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.29223324563
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 2.292233245631785, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.29)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.70503221978
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.7050322197753756, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.71)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 1.56046823206
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 1.5604682320599055, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 1.56)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 1.44076627012
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.4407662701216366, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.44)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: 0.567692467099
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'right'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 0.5676924670988189, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'right')
Agent drove forward instead of left. (rewarded 0.57)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.03935295061
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.0393529506105026, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.04)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.53109049693
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 13, 'action': None, 'reward': 1.531090496934443, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.53)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: right, reward: 0.467316789555
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 0.46731678955479583, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.47)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 1.60438659188
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.604386591884207, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.60)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 1.94463185354
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 1.9446318535396654, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.94)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 2.0899796672
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': 2.0899796671989934, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 2.09)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.05854888308
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 1.058548883082976, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.06)
37% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 225
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (7, 6), deadline = 20
Simulating trial. . .
epsilon = 0.1054; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: left, reward: 2.15688351998
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.156883519975618, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.16)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 2.55314740659
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.553147406587839, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.55)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 1.280789183
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.2807891829996343, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.28)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: forward, reward: 2.92198747147
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.9219874714658, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.92)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 1.41690886438
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.4169088643785321, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.42)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 2.8933540611
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.893354061097056, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.89)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: forward, reward: 1.83861016463
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.8386101646301676, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.84)
65% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 226
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (6, 7), deadline = 20
Simulating trial. . .
epsilon = 0.1044; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1044; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1044; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 1.44646598795
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.4464659879531903, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.45)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 2.60280456002
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.602804560019231, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.60)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 2.20773498973
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.207734989726454, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.21)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 2.25011894601
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.250118946008861, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.25)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 1.32357824895
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.3235782489457222, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.32)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 1.42003702348
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.4200370234751643, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.42)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: left, reward: 1.03386916591
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.0338691659096932, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.03)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.2817381115
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.281738111504228, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 2.28)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.76180257933
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.7618025793260546, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.76)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.49766689716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.4976668971637617, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.50)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.16494185769
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.1649418576867157, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.16)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 0.869624522005
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'right'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.8696245220049306, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'right')
Agent drove right instead of forward. (rewarded 0.87)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 1.3516412781
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.3516412781008598, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.35)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 2.1584480732
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 2.15844807319656, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.16)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 0.703726467365
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.7037264673652728, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 0.70)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: forward, reward: -9.54165459568
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -9.541654595679837, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.54)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: forward, reward: 1.3911662279
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.3911662278972041, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', None)
Agent drove forward instead of left. (rewarded 1.39)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 2.29152412878
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 3, 't': 17, 'action': None, 'reward': 2.291524128784296, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.29)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 1.53367112641
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.5336711264095924, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.53)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 1.43161604243
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.4316160424312552, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.43)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 227
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (7, 4), deadline = 20
Simulating trial. . .
epsilon = 0.1033; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1033; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1033; alpha = 0.1000
Simulating trial. . .
epsilon = 0.1033; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 2.14083905853
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.1408390585261245, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.14)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 1.76348531305
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.7634853130508423, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.76)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 1.22095565321
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.2209556532137749, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.22)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: left, reward: 1.79453165472
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 1.7945316547208603, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', 'left')
Agent drove left instead of forward. (rewarded 1.79)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.31103923501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.3110392350115754, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.31)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: -5.90537925083
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 15, 't': 5, 'action': None, 'reward': -5.905379250833465, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.91)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: right, reward: 1.67125371758
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'left', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.6712537175781477, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'left', None)
Agent followed the waypoint right. (rewarded 1.67)
65% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 228
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (7, 2), deadline = 30
Simulating trial. . .
epsilon = 0.1023; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.08778211528
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.087782115277769, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.09)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: 0.325118428716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 0.3251184287155936, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent drove right instead of forward. (rewarded 0.33)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: 2.02528529739
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 28, 't': 2, 'action': 'left', 'reward': 2.0252852973947633, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.03)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.02888122056
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.0288812205572926, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.03)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.89569198651
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.8956919865053905, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.90)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 2.77852076424
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 2.7785207642419403, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.78)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.37904407911
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.3790440791095016, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.38)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 1.00133765684
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 1.001337656838203, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.00)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.42151688685
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.4215168868546895, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.42)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 2.25173868887
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 9, 'action': None, 'reward': 2.251738688871497, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.25)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.42491870662
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.424918706620447, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.42)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 1.78108160758
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.7810816075822857, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.78)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: left, reward: 1.98179294856
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 18, 't': 12, 'action': 'left', 'reward': 1.9817929485614818, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.98)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: 0.724876695316
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 0.7248766953160164, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove left instead of forward. (rewarded 0.72)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: 2.37886433446
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 2.378864334460951, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.38)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: right, reward: 1.75645967931
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.7564596793109446, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.76)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: left, reward: 2.59449129542
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 2.594491295415294, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.59)
43% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 229
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (6, 2), deadline = 20
Simulating trial. . .
epsilon = 0.1013; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: forward, reward: 2.53864668637
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.5386466863690034, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.54)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: forward, reward: 1.63262852337
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.6326285233729725, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.63)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 0.873039028559
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 0.8730390285587569, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 0.87)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 2.79409875031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.7940987503130357, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.79)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 1.84137930653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.8413793065296682, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.84)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 2.11333676859
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.1133367685925473, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.11)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 0.0762871593185
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.07628715931848618, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 0.08)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.75237710222
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.752377102219921, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.75)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 0.877623437814
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 0.8776234378136869, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.88)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: right, reward: 2.33087885498
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 2.330878854981324, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.33)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 0.0121282215284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 0.012128221528351246, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.01)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 2.59799533246
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.5979953324630882, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.60)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 0.870326491712
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.8703264917120896, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.87)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 0.666842445266
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.6668424452659573, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.67)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 1.97958415547
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.9795841554717597, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.98)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 0.368245818786
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'forward'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 0.36824581878622054, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.37)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: left, reward: 0.521230872074
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 0.5212308720740062, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent followed the waypoint left. (rewarded 0.52)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: 0.891528274254
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.8915282742543972, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.89)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 0.487673207265
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.4876732072645522, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 0.49)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 0.200914821813
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.20091482181331788, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.20)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 230
\-------------------------
Environment.reset(): Trial set up with start = (2, 7), destination = (8, 3), deadline = 20
Simulating trial. . .
epsilon = 0.1003; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: left, reward: 2.90200242
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.902002419999287, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.90)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 1.12175503749
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.1217550374892338, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 1.12)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: left, reward: 2.71009906632
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 2.710099066317885, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 2.71)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 2.65431689065
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.654316890651929, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.65)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: forward, reward: 2.77408207286
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.774082072864867, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.77)
75% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 231
\-------------------------
Environment.reset(): Trial set up with start = (4, 3), destination = (8, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0993; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: forward, reward: 1.16677770638
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.1667777063784195, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.17)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 1.04154988028
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'right'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.0415498802817262, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 1.04)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 1.17215084836
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.1721508483568488, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.17)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 0.565731256108
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 0.5657312561076074, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.57)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 2.05966229572
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.059662295721175, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.06)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 1.5575463811
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.5575463810960026, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.56)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.20120657864
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'right'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.2012065786393373, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'right')
Agent properly idled at a red light. (rewarded 2.20)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.95326212455
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.9532621245517436, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.95)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 0.618611793063
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.61861179306342, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.62)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: left, reward: 2.49684183264
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 2.4968418326422137, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.50)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 232
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (8, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0983; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0983; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0983; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0983; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 1.70649505362
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.7064950536233827, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.71)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: 2.32256708336
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 2.3225670833606125, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.32)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 1.06108501971
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.0610850197110904, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.06)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 1.30635535641
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.3063553564118535, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.31)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 1.14481320259
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.1448132025890625, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.14)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: left, reward: 0.190186670733
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 0.19018667073296402, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.19)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 1.26204429585
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.2620442958528026, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.26)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 1.9829446408
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.9829446407960947, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.98)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 233
\-------------------------
Environment.reset(): Trial set up with start = (7, 4), destination = (2, 7), deadline = 30
Simulating trial. . .
epsilon = 0.0973; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 2.79397744682
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 2.7939774468231233, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.79)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: left, reward: 1.63471395945
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 29, 't': 1, 'action': 'left', 'reward': 1.6347139594470712, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent drove left instead of right. (rewarded 1.63)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 1.97914268334
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.9791426833364374, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.98)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 1.0077182364
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.007718236397096, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.01)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 1.28011768619
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 1.2801176861946855, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.28)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 2.74161091351
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.7416109135065208, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.74)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: 1.11671883614
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 1.116718836139154, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.12)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: left, reward: 0.456330128258
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 0.45633012825834907, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.46)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 2.38248741234
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 2.3824874123356854, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.38)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 2.71493399713
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 9, 'action': None, 'reward': 2.71493399712626, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.71)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 2.16987183082
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.1698718308191993, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.17)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 1.17722510936
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 1.1772251093611934, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove right instead of forward. (rewarded 1.18)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: left, reward: 2.03665815889
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 18, 't': 12, 'action': 'left', 'reward': 2.0366581588891917, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 2.04)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.66293615147
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 13, 'action': None, 'reward': 1.662936151469987, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.66)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 2.07952716286
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 2.079527162857673, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.08)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 0.420655180531
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 0.4206551805309665, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.42)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: left, reward: 1.72678172349
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 1.726781723486627, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.73)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 2.43871594608
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 13, 't': 17, 'action': None, 'reward': 2.438715946079111, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.44)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.90809426709
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.9080942670926566, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.91)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 0.865685274786
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 19, 'action': None, 'reward': 0.8656852747860955, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.87)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 0.818142194586
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 20, 'action': None, 'reward': 0.8181421945858134, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.82)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: 1.56622504582
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'left', 'reward': 1.566225045818964, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.57)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: 1.07239856334
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': 1.0723985633429698, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.07)
23% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 234
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (6, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0963; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 1.72262744666
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.722627446655153, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 1.72)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 2.93592982103
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.9359298210267317, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.94)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 2.57980007458
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.579800074584492, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.58)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 1.38502771284
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.3850277128416728, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.39)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 2.78088268159
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.7808826815854, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.78)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 1.45822893173
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.4582289317311175, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.46)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: left, reward: 1.44360907877
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.4436090787730567, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.44)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 1.00336364178
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.0033636417769056, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.00)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 1.76901396766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.7690139676577865, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.77)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 1.17832648275
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.17832648274835, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.18)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 2.23106815101
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.231068151006677, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.23)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 0.710777975521
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': None, 'reward': 0.7107779755213915, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.71)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: left, reward: 1.08263871915
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 13, 't': 12, 'action': 'left', 'reward': 1.0826387191517448, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.08)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: right, reward: 0.886787035694
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 0.8867870356941665, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.89)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 1.15733838406
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.1573383840552067, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.16)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: left, reward: 2.04431134836
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 2.0443113483559086, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.04)
36% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 235
\-------------------------
Environment.reset(): Trial set up with start = (8, 7), destination = (3, 4), deadline = 30
Simulating trial. . .
epsilon = 0.0954; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 1.85539011508
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.8553901150786938, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'left')
Agent followed the waypoint right. (rewarded 1.86)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 1.29693794168
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.2969379416807163, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.30)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 1.761772072
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.7617720720013323, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.76)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 1.99808683272
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 1.9980868327185164, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.00)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 2.95139525835
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.951395258347385, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.95)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: 2.93785897447
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 2.9378589744682353, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.94)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: 1.69211393932
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 1.6921139393235358, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 1.69)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: -10.3393395879
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': -10.33933958791351, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.34)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 2.90123532746
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'right'), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.9012353274622384, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'right')
Agent properly idled at a red light. (rewarded 2.90)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: 1.78507165346
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.7850716534572983, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.79)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: -4.80085198436
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 20, 't': 10, 'action': None, 'reward': -4.800851984361544, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.80)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 1.14544757065
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 1.1454475706468226, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove right instead of forward. (rewarded 1.15)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.63141692081
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.6314169208129297, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.63)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 1.66716792057
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.6671679205702772, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.67)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 0.389585208368
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': 0.3895852083683655, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 0.39)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: -40.3249803817
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'right'), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': -40.32498038174898, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'right')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.32)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 2.64368878434
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 16, 'action': None, 'reward': 2.643688784337768, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.64)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: left, reward: 1.3008289041
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 17, 'action': 'left', 'reward': 1.3008289040961685, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.30)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 0.995634328792
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 12, 't': 18, 'action': None, 'reward': 0.9956343287920442, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.00)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: forward, reward: 1.09252005473
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': 1.0925200547291558, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.09)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 2.41036085427
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 10, 't': 20, 'action': None, 'reward': 2.410360854265285, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.41)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: 0.907586897318
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 21, 'action': 'left', 'reward': 0.9075868973181767, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.91)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 0.517390740551
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 22, 'action': None, 'reward': 0.5173907405505971, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.52)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: 2.03266192339
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 7, 't': 23, 'action': 'forward', 'reward': 2.0326619233867245, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.03)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: left, reward: 0.75582349767
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 6, 't': 24, 'action': 'left', 'reward': 0.7558234976700913, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.76)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: 0.0142504716509
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 5, 't': 25, 'action': 'forward', 'reward': 0.014250471650875784, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 0.01)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: 1.9427380874
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'left'), 'deadline': 4, 't': 26, 'action': 'right', 'reward': 1.9427380874040343, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'left')
Agent followed the waypoint right. (rewarded 1.94)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 1.16458197772
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 27, 'action': 'right', 'reward': 1.1645819777196202, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.16)
7% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 236
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (2, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0944; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0944; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0944; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 2.99658497803
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.996584978033215, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 3.00)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 1.40054273528
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.4005427352847137, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.40)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.35076941136
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.3507694113611641, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.35)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 0.967545902806
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 0.9675459028064259, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.97)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: -9.71342281346
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': -9.713422813459813, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.71)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: -4.29101201995
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': -4.2910120199457165, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.29)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: 1.88212050709
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.8821205070891796, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.88)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: 1.33352790462
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.333527904618159, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.33)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 237
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (8, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0935; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.20663294559
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.2066329455853813, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.21)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.48976402561
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.489764025607284, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.49)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.40967263113
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'right', 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.409672631134799, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.41)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.23052472202
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.2305247220163213, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.23)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.95724631386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.9572463138576712, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.96)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: left, reward: -10.377619401
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': -10.377619401019995, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.38)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: 0.620645949419
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'forward'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 0.6206459494185625, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.62)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: forward, reward: 0.799586101072
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 0.7995861010723775, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.80)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: 1.63891759156
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.6389175915610399, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.64)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 2.27044242878
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 2.2704424287832996, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.27)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 0.335194413286
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 0.33519441328557786, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.34)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.3441256023
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.3441256023048926, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.34)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.48207643847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.482076438465315, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.48)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 0.738809901794
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'right'), 'deadline': 7, 't': 13, 'action': None, 'reward': 0.7388099017939018, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'right')
Agent properly idled at a red light. (rewarded 0.74)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 0.0733429769603
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 0.07334297696025216, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.07)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: left, reward: 2.39663047867
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 2.396630478668905, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.40)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 1.75450275757
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.7545027575699619, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.75)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: left, reward: 1.62399577733
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 1.6239957773291522, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 1.62)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: left, reward: -10.8085075958
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -10.808507595771005, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.81)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: forward, reward: 1.76883117203
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 1.76883117203047, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.77)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 238
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (1, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0926; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: 1.31595971579
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.3159597157852874, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.32)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: 1.36813769634
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.368137696340282, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.37)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: 2.33992443813
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.339924438127598, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.34)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: 1.195490234
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.1954902340037794, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.20)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: 2.41603832809
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.416038328091845, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.42)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: 1.19285961791
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.1928596179092361, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.19)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 1.25167783084
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.2516778308420786, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.25)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 1.94377136453
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.9437713645289845, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 1.94)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 0.129525081005
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.1295250810052907, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.13)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 2.55697193754
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.5569719375441444, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 2.56)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 2.57316190678
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.573161906781662, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.57)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 1.85469228686
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.8546922868553406, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.85)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 1.92643775114
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.9264377511417636, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.93)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: left, reward: 0.991727438778
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 0.9917274387776054, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.99)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 0.910885868596
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 0.9108858685955079, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.91)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 239
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (6, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0916; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: 1.90874334135
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.9087433413511143, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.91)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: left, reward: 1.35679730719
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.35679730719102, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove left instead of right. (rewarded 1.36)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 2.09745805378
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.097458053784492, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.10)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: forward, reward: 0.968319852322
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 0.9683198523217933, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.97)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: None, reward: 2.65790303185
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.6579030318476162, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.66)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: None, reward: 1.6265995096
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.6265995096022594, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.63)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: None, reward: 2.19047415812
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.190474158120738, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.19)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: left, reward: 0.934816997018
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 0.9348169970177671, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.93)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.81577747671
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.8157774767132187, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.82)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.57592111993
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.5759211199317416, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.58)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: 1.15840625023
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.1584062502303558, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.16)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 2.41460307833
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 2.4146030783295505, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.41)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: right, reward: 1.62198285354
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.6219828535364187, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.62)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: right, reward: 0.151765963299
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.15176596329863778, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.15)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: 2.36585155358
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 2.365851553579784, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 2.37)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 2.13125810237
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.131258102368302, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.13)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: left, reward: 0.430583123764
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 0.4305831237644253, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 0.43)
15% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 240
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (6, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0907; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0907; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0907; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0907; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: 1.08412943049
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.0841294304872497, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.08)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: left, reward: 0.754185529257
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 0.7541855292567602, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', None)
Agent drove left instead of right. (rewarded 0.75)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: 0.470335728808
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.4703357288079092, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent drove right instead of left. (rewarded 0.47)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: 0.952943915524
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 0.9529439155237417, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 0.95)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.13869706112
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.1386970611232885, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.14)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 1.85578450756
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.8557845075571129, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.86)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 1.46439082296
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.4643908229631617, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.46)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 1.96499728433
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.964997284332226, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.96)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 2.22349964093
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 2.2234996409295746, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.22)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: 1.0934576253
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.0934576252965933, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.09)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: left, reward: 1.17449216996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.1744921699645934, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 1.17)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: forward, reward: 1.27975328414
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.2797532841381012, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.28)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: left, reward: -0.213780951368
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -0.21378095136841435, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded -0.21)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 0.915089595965
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.9150895959651683, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove forward instead of right. (rewarded 0.92)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: -9.96685875479
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'right', 'left'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': -9.966858754789367, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', 'left')
Agent attempted driving forward through a red light. (rewarded -9.97)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 2.15450198344
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 2.1545019834422456, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 2.15)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: forward, reward: 0.580207401655
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 0.5802074016553427, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.58)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 1.6944057943
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.694405794302437, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.69)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 0.69821224301
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.6982122430103752, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.70)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 1.95223765921
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'left'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 1.952237659213515, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'left', 'left', 'left')
Agent followed the waypoint right. (rewarded 1.95)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 241
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (8, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0898; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 2.13070899398
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.130708993979092, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', 'left')
Agent properly idled at a red light. (rewarded 2.13)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 2.50749475562
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.507494755624454, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.51)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.56302437379
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.563024373794359, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 1.56)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: 1.10183058773
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.101830587729037, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 1.10)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 2.78734180793
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.787341807927379, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.79)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 1.89861973011
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.8986197301147045, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.90)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: forward, reward: 1.12180344346
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.121803443464811, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove forward instead of left. (rewarded 1.12)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 2.86048958745
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.8604895874467173, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.86)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 1.14944110759
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.1494411075932383, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.15)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 2.30089656494
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.3008965649374464, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.30)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: left, reward: 2.60286467619
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 2.6028646761936365, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.60)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: -5.48363437005
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', None), 'deadline': 9, 't': 11, 'action': None, 'reward': -5.483634370048009, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.48)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 1.79405909204
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.7940590920400215, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.79)
35% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 242
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (5, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0889; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0889; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: left, reward: 2.96042053977
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.9604205397738763, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.96)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: left, reward: 1.66726327997
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.6672632799720932, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.67)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: forward, reward: 1.86018733281
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.8601873328072518, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 1.86)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 1.53616141993
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.5361614199281861, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.54)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: forward, reward: 2.9111300589
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.911130058898504, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.91)
75% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 243
\-------------------------
Environment.reset(): Trial set up with start = (2, 2), destination = (6, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0880; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 0.369181146673
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'right'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 0.3691811466726709, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'right')
Agent drove right instead of forward. (rewarded 0.37)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: left, reward: 2.1073790498
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'left'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 2.1073790498034457, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'left')
Agent followed the waypoint left. (rewarded 2.11)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: left, reward: 1.30384369098
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 1.303843690977734, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.30)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 1.81583946856
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.8158394685619503, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.82)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: left, reward: 0.0236782998872
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 0.023678299887227028, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.02)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 1.31172049012
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.3117204901200166, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.31)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: left, reward: 1.29091920009
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.2909192000948568, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 1.29)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 1.86484621236
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.8648462123555747, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.86)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: 2.06413700112
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 2.0641370011202045, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 2.06)
64% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 244
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (4, 5), deadline = 35
Simulating trial. . .
epsilon = 0.0872; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 2.81880090013
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'right'), 'deadline': 35, 't': 0, 'action': None, 'reward': 2.8188009001306504, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 2.82)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.85283923878
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 34, 't': 1, 'action': None, 'reward': 1.8528392387787556, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.85)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.30157089669
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 33, 't': 2, 'action': None, 'reward': 1.3015708966946422, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.30)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: -10.6398160532
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 32, 't': 3, 'action': 'left', 'reward': -10.63981605322436, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.64)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 0.459172422421
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 31, 't': 4, 'action': 'forward', 'reward': 0.4591724224212068, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent drove forward instead of right. (rewarded 0.46)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: 2.35589283831
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 30, 't': 5, 'action': 'forward', 'reward': 2.3558928383071147, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.36)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: 2.92105452844
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 29, 't': 6, 'action': 'forward', 'reward': 2.9210545284445173, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.92)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: -20.5707334617
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', None), 'deadline': 28, 't': 7, 'action': 'right', 'reward': -20.570733461736808, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.57)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 2.02117196833
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 27, 't': 8, 'action': None, 'reward': 2.0211719683288782, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.02)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 1.74973787809
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 26, 't': 9, 'action': None, 'reward': 1.7497378780869657, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.75)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: 2.35905733689
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 10, 'action': 'forward', 'reward': 2.3590573368903556, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.36)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 1.03428741604
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 11, 'action': None, 'reward': 1.0342874160384397, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.03)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 2.08820337565
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 12, 'action': None, 'reward': 2.0882033756533565, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.09)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 1.55228500085
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 22, 't': 13, 'action': 'right', 'reward': 1.552285000850947, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.55)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: -5.25882252376
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', None), 'deadline': 21, 't': 14, 'action': None, 'reward': -5.2588225237628645, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.26)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: 1.30435504161
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 20, 't': 15, 'action': 'forward', 'reward': 1.3043550416103389, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.30)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: 1.67253062858
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 19, 't': 16, 'action': 'left', 'reward': 1.672530628580431, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.67)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: -39.8698390636
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'right', 'forward', 'left'), 'deadline': 18, 't': 17, 'action': 'left', 'reward': -39.869839063648726, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.87)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: 1.26620946978
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 17, 't': 18, 'action': 'right', 'reward': 1.2662094697801596, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.27)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: 1.64941880655
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 16, 't': 19, 'action': 'left', 'reward': 1.6494188065454485, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 1.65)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: right, reward: 1.88411909031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 15, 't': 20, 'action': 'right', 'reward': 1.884119090313263, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.88)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 0.975800288512
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 14, 't': 21, 'action': 'right', 'reward': 0.9758002885118191, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.98)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 0.745052164682
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 22, 'action': None, 'reward': 0.7450521646818149, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.75)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: 2.48308803944
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 12, 't': 23, 'action': 'forward', 'reward': 2.4830880394388535, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.48)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 2.12723460144
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'right', 'right'), 'deadline': 11, 't': 24, 'action': None, 'reward': 2.1272346014392487, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', 'right')
Agent properly idled at a red light. (rewarded 2.13)
29% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: right, reward: 2.36795763172
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 25, 'action': 'right', 'reward': 2.3679576317157083, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.37)
26% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 245
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (5, 7), deadline = 35
Simulating trial. . .
epsilon = 0.0863; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 2.26430742708
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'left'), 'deadline': 35, 't': 0, 'action': None, 'reward': 2.2643074270819445, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.26)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 2.00753857498
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 34, 't': 1, 'action': None, 'reward': 2.007538574983872, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.01)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 1.13620990336
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 33, 't': 2, 'action': 'right', 'reward': 1.1362099033637187, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.14)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: 2.92002732143
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 32, 't': 3, 'action': 'forward', 'reward': 2.9200273214294508, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.92)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: 2.11078577526
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 31, 't': 4, 'action': 'forward', 'reward': 2.1107857752587056, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.11)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 2.36511690849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 30, 't': 5, 'action': None, 'reward': 2.365116908486339, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.37)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 2.24600026441
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 29, 't': 6, 'action': None, 'reward': 2.24600026441239, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.25)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 2.49362004259
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 28, 't': 7, 'action': None, 'reward': 2.493620042594232, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.49)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 1.14993135481
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 8, 'action': None, 'reward': 1.1499313548140657, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.15)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: forward, reward: 1.01112150748
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 26, 't': 9, 'action': 'forward', 'reward': 1.0111215074848146, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.01)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: forward, reward: 1.69585652658
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 25, 't': 10, 'action': 'forward', 'reward': 1.695856526581677, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.70)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: left, reward: 1.71827052564
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 24, 't': 11, 'action': 'left', 'reward': 1.7182705256370359, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.72)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: 1.43084823567
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 23, 't': 12, 'action': 'left', 'reward': 1.4308482356700363, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.43)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 1.50042875796
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 22, 't': 13, 'action': None, 'reward': 1.5004287579565982, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.50)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 2.42694745417
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 21, 't': 14, 'action': None, 'reward': 2.4269474541714517, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.43)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 2.10343736446
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 20, 't': 15, 'action': None, 'reward': 2.103437364455001, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.10)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 1.22364581979
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 16, 'action': None, 'reward': 1.2236458197932536, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.22)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: right, reward: 1.77973584693
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 18, 't': 17, 'action': 'right', 'reward': 1.7797358469261064, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.78)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: None, reward: 2.59972098119
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 18, 'action': None, 'reward': 2.5997209811933537, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.60)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: left, reward: 0.0308993318718
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 16, 't': 19, 'action': 'left', 'reward': 0.030899331871806823, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove left instead of forward. (rewarded 0.03)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 2.0180072424
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 15, 't': 20, 'action': 'right', 'reward': 2.018007242395629, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.02)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: right, reward: 0.971184163912
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 14, 't': 21, 'action': 'right', 'reward': 0.9711841639121801, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 0.97)
37% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 246
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (4, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0854; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: 2.45747578694
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.45747578693721, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.46)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 2.41964306502
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 2.41964306501507, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 2.42)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 2.55990433633
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 2.5599043363295557, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.56)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 2.2403384051
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.2403384050974684, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.24)
80% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 247
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (6, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0846; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: 2.16805117754
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.168051177540434, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.17)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.0588235252
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.058823525204968, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.06)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 1.10709198195
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.107091981953836, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 1.11)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.66716081521
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.667160815213697, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.67)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: left, reward: 1.08945030565
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.0894503056507598, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.09)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 0.979990839011
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 0.9799908390113061, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.98)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.62187173948
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.6218717394783078, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.62)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.41926061616
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.4192606161600407, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.42)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.80974606153
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.8097460615329903, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.81)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.13027602491
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.130276024907511, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.13)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 0.44601637785
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.4460163778499099, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent drove right instead of left. (rewarded 0.45)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: right, reward: 0.74854625078
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.7485462507804659, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.75)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 2.02614898734
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.02614898734316, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.03)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: right, reward: 2.40283748305
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 2.4028374830505657, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.40)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: forward, reward: -0.0541072547327
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': -0.054107254732651655, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded -0.05)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: forward, reward: 1.22268701287
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 1.2226870128702483, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.22)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 1.83911701175
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.8391170117490432, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.84)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 1.20011893909
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 1.2001189390918756, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.20)
10% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 248
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (4, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0837; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: left, reward: 2.86163009156
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.861630091560232, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.86)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.43483148657
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.434831486569324, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 1.43)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 1.20805640791
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.2080564079056149, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.21)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: forward, reward: 2.72331599539
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.7233159953912844, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.72)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: 2.83706667697
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.8370666769737705, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.84)
75% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 249
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (7, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0829; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 2.85927890121
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.8592789012078663, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.86)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 1.48968972793
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.4896897279342567, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.49)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 1.5014074469
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.501407446895251, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove right instead of left. (rewarded 1.50)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 1.8612283821
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.861228382099392, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.86)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: left, reward: 1.41905989616
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.4190598961576053, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 1.42)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: 2.38459091807
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.3845909180706424, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.38)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 1.87673804264
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.8767380426434686, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.88)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: left, reward: 1.38902804711
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.3890280471100538, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', 'left')
Agent drove left instead of forward. (rewarded 1.39)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 2.48964155619
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.4896415561920593, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.49)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: 2.48271114904
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 2.4827111490392966, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.48)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: 1.98825443605
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.9882544360515655, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.99)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: forward, reward: 1.57252113011
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.5725211301108324, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.57)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 250
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (6, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0821; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 1.98190516907
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.9819051690710947, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.98)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 1.9700251066
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.9700251065951586, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.97)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 2.85986441169
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.8598644116905634, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.86)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: 2.15345132111
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 2.153451321111132, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.15)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 0.307885447781
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.30788544778066995, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.31)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: 1.96809526538
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.9680952653828567, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 1.97)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 1.76213073723
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'right'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.762130737227933, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'right')
Agent properly idled at a red light. (rewarded 1.76)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.19144188696
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'forward'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.191441886963017, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.19)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.40277757303
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.402777573033705, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.40)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: -10.4756393974
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': -10.47563939735866, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -10.48)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 1.81870238428
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.8187023842819519, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.82)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 2.46596824812
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 2.4659682481176066, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.47)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.2938360909
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.2938360908961282, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.29)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: 2.36305904683
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 2.3630590468331185, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.36)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: forward, reward: 1.58184718382
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 1.5818471838244643, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.58)
25% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 251
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (6, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0813; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 1.74055177489
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.7405517748927812, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.74)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: forward, reward: 1.43619189414
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.4361918941356864, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 1.44)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: 2.87386658338
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.8738665833849164, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.87)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 2.33070262824
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.330702628238906, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.33)
80% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 252
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (2, 3), deadline = 30
Simulating trial. . .
epsilon = 0.0805; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 1.37648494862
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.3764849486158672, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.38)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: left, reward: 1.02009244159
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'right', None), 'deadline': 29, 't': 1, 'action': 'left', 'reward': 1.020092441593789, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'right', None)
Agent drove left instead of right. (rewarded 1.02)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: 1.20647238739
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 1.2064723873912229, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.21)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 2.54700000271
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 2.5470000027121023, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.55)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 2.62147940278
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.621479402781574, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.62)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.60087172189
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.600871721888193, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.60)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: 2.76442028839
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 2.764420288388638, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.76)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: -10.0854931647
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': -10.085493164659702, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.09)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 0.927947917853
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 22, 't': 8, 'action': None, 'reward': 0.9279479178529726, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 0.93)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 2.478425165
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 2.478425164998399, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.48)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 2.1634209763
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'left'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 2.163420976302019, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'left')
Agent followed the waypoint right. (rewarded 2.16)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: left, reward: 0.509367178396
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 0.5093671783963639, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.51)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.31087912604
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.3108791260445003, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.31)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 0.860373778572
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 0.8603737785719099, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.86)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 1.64601577565
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.646015775647156, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.65)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 253
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (6, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0797; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.0121065151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.0121065151015696, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 1.01)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.60839777208
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.6083977720782927, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.61)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.04058364411
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.0405836441103724, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.04)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.55520305943
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.5552030594299264, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.56)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 1.53247428441
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.5324742844099086, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.53)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.40657907714
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.4065790771408346, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 1.41)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 0.925770116068
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 0.9257701160680074, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.93)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: -0.132747308411
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -0.1327473084110189, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded -0.13)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.55475248493
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.554752484925939, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.55)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 0.0787391114112
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.0787391114111804, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 0.08)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 2.68844718477
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.688447184768048, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.69)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: -0.019914536054
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': -0.019914536054013388, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded -0.02)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 1.42165565272
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.4216556527181445, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.42)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: left, reward: 2.41605930657
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 2.4160593065717704, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.42)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 1.6757407901
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.675740790095795, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.68)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 2.42960857354
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.429608573542387, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.43)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 1.53310163083
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.5331016308305443, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.53)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: 2.08327327745
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 2.0832732774523333, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 2.08)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: forward, reward: 2.01559893022
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 2.015598930221987, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 2.02)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: 0.574748410835
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 0.5747484108349502, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.57)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 254
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (7, 3), deadline = 35
Simulating trial. . .
epsilon = 0.0789; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 2.93704273275
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 35, 't': 0, 'action': None, 'reward': 2.9370427327481528, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.94)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.20943712359
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'right'), 'deadline': 34, 't': 1, 'action': None, 'reward': 1.2094371235914523, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 1.21)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.63446635145
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', None), 'deadline': 33, 't': 2, 'action': None, 'reward': 1.6344663514535167, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 1.63)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 2.76857202875
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 32, 't': 3, 'action': 'right', 'reward': 2.7685720287479345, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.77)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 2.80059143577
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 31, 't': 4, 'action': 'right', 'reward': 2.8005914357733177, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.80)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 2.34297891608
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 30, 't': 5, 'action': 'forward', 'reward': 2.3429789160809804, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.34)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 2.41851968529
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 29, 't': 6, 'action': 'forward', 'reward': 2.4185196852894606, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.42)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.06766945039
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 28, 't': 7, 'action': None, 'reward': 1.0676694503890158, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.07)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 2.58277428893
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 8, 'action': None, 'reward': 2.5827742889251004, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.58)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.52765986725
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 26, 't': 9, 'action': None, 'reward': 1.5276598672472836, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.53)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.91072498461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'left'), 'deadline': 25, 't': 10, 'action': None, 'reward': 1.91072498461143, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.91)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 1.13274172682
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', 'left'), 'deadline': 24, 't': 11, 'action': 'forward', 'reward': 1.1327417268228834, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', 'left')
Agent followed the waypoint forward. (rewarded 1.13)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.03131407208
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 23, 't': 12, 'action': None, 'reward': 2.0313140720757703, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.03)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 1.38035083978
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 22, 't': 13, 'action': None, 'reward': 1.3803508397830486, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.38)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.50560586556
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'right'), 'deadline': 21, 't': 14, 'action': None, 'reward': 2.5056058655565687, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'right')
Agent properly idled at a red light. (rewarded 2.51)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.7328816538
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 15, 'action': None, 'reward': 2.7328816538013934, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.73)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: left, reward: 1.48885253503
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 16, 'action': 'left', 'reward': 1.4888525350342874, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.49)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 2.35265313389
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 18, 't': 17, 'action': None, 'reward': 2.3526531338931194, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.35)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 0.940506947264
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 17, 't': 18, 'action': None, 'reward': 0.940506947264131, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.94)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: forward, reward: 1.80692090022
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 16, 't': 19, 'action': 'forward', 'reward': 1.8069209002235176, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.81)
43% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 255
\-------------------------
Environment.reset(): Trial set up with start = (7, 2), destination = (5, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0781; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.52648077231
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.526480772306037, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.53)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.15173245254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.1517324525445876, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.15)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.73198451874
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.731984518741969, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.73)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: 2.39796604462
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.397966044617613, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.40)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.53105482637
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.5310548263706634, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.53)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.28238684631
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.282386846314624, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.28)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: 2.23431315132
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', 'left'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.2343131513247867, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', 'left')
Agent followed the waypoint forward. (rewarded 2.23)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 2.15708230757
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 2.1570823075734324, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.16)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: forward, reward: 2.52196469011
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 2.5219646901114885, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.52)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 1.66242471642
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.6624247164210328, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.66)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 1.8139101122
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.813910112196191, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.81)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 2.40395433232
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.4039543323229466, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.40)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.3427320145
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.3427320145007848, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.34)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: left, reward: 2.60064766608
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 2.600647666075936, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.60)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 0.815951936548
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 0.8159519365481116, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.82)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: left, reward: 2.27759484195
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 2.277594841950084, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.28)
36% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 256
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (7, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0773; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: -40.1842915215
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'left'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -40.184291521544424, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.18)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: left, reward: -39.5732637063
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'left'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -39.57326370627356, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.57)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 1.87445991806
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.874459918056299, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.87)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 2.8509195349
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.8509195348981633, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.85)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: left, reward: 2.28956191767
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 2.28956191767189, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.29)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 1.49268017852
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.4926801785183257, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.49)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 1.09511341972
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.0951134197183796, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.10)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 2.1032161368
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.1032161367955737, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.10)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 2.02018954146
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.0201895414602413, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.02)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: left, reward: 2.03281408455
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 2.032814084554259, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.03)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 1.82855421912
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.828554219119604, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.83)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 2.57503393658
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'right'), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.5750339365813444, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 2.58)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 0.957476575076
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 0.957476575075743, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.96)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: forward, reward: 2.55113419355
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 2.5511341935508343, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.55)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 1.41314215515
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.4131421551546093, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.41)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 2.20879915002
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.2087991500173416, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.21)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: 2.34292687896
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 2.342926878962575, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.34)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: right, reward: 1.12487956447
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.1248795644741159, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 1.12)
28% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 257
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (1, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0765; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 2.94383910237
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.9438391023745325, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.94)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: -10.1685876918
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -10.16858769175544, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -10.17)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.20299441147
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.2029944114741593, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.20)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: -4.49407255014
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 17, 't': 3, 'action': None, 'reward': -4.4940725501432635, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.49)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 1.61550343311
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.6155034331073748, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.62)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: 2.54178413281
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 2.541784132812065, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.54)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 2.80405387976
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.804053879761671, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.80)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 2.07318112988
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.073181129881962, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.07)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.04583919044
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.045839190442115, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 1.05)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.01549713647
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.0154971364714718, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.02)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 2.36521694728
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 2.3652169472765157, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.37)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 258
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (6, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0758; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 2.90282386676
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.9028238667608477, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.90)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 1.32846104511
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.3284610451074426, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.33)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 2.50678544391
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.506785443909059, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.51)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: 1.59905522495
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 1.599055224950252, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.60)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: -10.4368414784
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -10.436841478417165, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -10.44)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 2.53312929702
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.533129297024148, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.53)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 1.36010365665
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.3601036566500126, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.36)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 1.47458514664
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.4745851466399982, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.47)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 2.05930809223
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.059308092231613, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.06)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: left, reward: 2.29710820961
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 2.2971082096120874, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.30)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 2.44433382823
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.444333828231055, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.44)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 1.43887123009
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.4388712300946793, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.44)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 2.12386986132
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.1238698613184117, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.12)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: forward, reward: 2.26844896923
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 2.2684489692274705, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.27)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: forward, reward: 1.9748466077
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 1.9748466077043734, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.97)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 259
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (5, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0750; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: forward, reward: 0.86775200371
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 0.8677520037103628, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 0.87)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 0.376803957103
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.3768039571030736, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 0.38)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.46186176143
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.4618617614300993, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.46)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.63216892782
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.6321689278162879, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.63)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.11390875854
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.1139087585435643, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.11)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.08160269954
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.0816026995388666, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.08)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: -0.0294166106958
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': -0.029416610695792045, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove right instead of forward. (rewarded -0.03)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 0.993110718812
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 0.9931107188118888, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.99)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 1.66444793023
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.66444793022624, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 1.66)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 2.54733469062
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.5473346906202936, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.55)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.42775451231
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.427754512310215, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.43)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 2.04144794417
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'right'), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.0414479441723072, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 2.04)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 0.747968968153
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.7479689681530184, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 0.75)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 0.970578764956
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.9705787649555393, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.97)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 1.17802895432
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 1.1780289543244074, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.18)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 0.786743691151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.7867436911511256, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.79)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 0.866042525211
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.8660425252106376, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 0.87)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: 0.512254889388
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.5122548893880545, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.51)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: left, reward: 1.18966953826
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': 1.189669538261412, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', None)
Agent drove left instead of right. (rewarded 1.19)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: None, reward: 1.01806399528
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.0180639952772017, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.02)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 260
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (3, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0743; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: forward, reward: 0.772847673839
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 0.7728476738394177, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 0.77)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 1.18385797573
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.1838579757315657, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.18)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: 0.701674402276
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.7016744022760625, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent drove right instead of left. (rewarded 0.70)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: 2.35018908462
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.3501890846235245, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.35)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: -39.7489661799
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -39.748966179920465, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.75)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.36212119199
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.3621211919887624, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.36)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.59131850933
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.5913185093265076, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.59)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 2.28126677859
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.2812667785855876, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'forward')
Agent followed the waypoint forward. (rewarded 2.28)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 1.53591174682
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.5359117468236492, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.54)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.07917376529
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.0791737652901932, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.08)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.2568101194
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.2568101193987014, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.26)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 1.25815929416
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.2581592941556583, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.26)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: left, reward: 0.401566906964
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 0.4015669069639918, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.40)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 1.64274392702
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.6427439270151534, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.64)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 1.47781964499
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.4778196449940455, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.48)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 1.89051396614
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.8905139661432586, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.89)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 261
\-------------------------
Environment.reset(): Trial set up with start = (4, 3), destination = (6, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0735; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: right, reward: 1.81885425443
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.8188542544332822, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove right instead of left. (rewarded 1.82)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 2.88671149356
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.886711493558732, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.89)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: right, reward: 2.95896911637
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.958969116365812, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 2.96)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: 2.53419345674
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.534193456739028, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.53)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 2.52027104831
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.5202710483090556, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.52)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.16427269927
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.164272699269168, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.16)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: forward, reward: 1.09931932829
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'forward'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.0993193282938292, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.10)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.04761515676
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.0476151567573018, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', None)
Agent properly idled at a red light. (rewarded 1.05)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.95946035073
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.9594603507271837, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.96)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: left, reward: 1.71098622552
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.7109862255213044, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.71)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 0.508577593787
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.5085775937871936, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 0.51)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: -40.834464002
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': -40.83446400202862, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.83)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.93768663442
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.937686634416491, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.94)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 1.98861153203
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.9886115320297006, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.99)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 1.18606039353
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.186060393530847, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.19)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 2.17158568015
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 2.171585680145335, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.17)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.08473085683
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 2.0847308568330183, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.08)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 0.564200938042
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'right'), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.5642009380418785, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'right')
Agent drove right instead of forward. (rewarded 0.56)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 0.367445322812
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.36744532281242615, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.37)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: 1.81377902137
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 1.8137790213699871, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.81)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 262
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (4, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0728; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 2.98279471278
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.9827947127780936, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.98)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.44573381717
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.4457338171659113, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.45)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 2.1576623273
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.1576623273007702, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.16)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.35613260015
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.356132600145189, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.36)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: 1.58219361107
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.582193611069084, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 1.58)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 2.33239779852
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.332397798523925, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.33)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 1.89576378233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.8957637823304838, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.90)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 2.00842293358
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.0084229335844865, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.01)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 2.06327174672
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.063271746722669, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.06)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: forward, reward: 2.75064294766
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 2.7506429476561065, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.75)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: left, reward: 0.575753159313
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 0.5757531593125286, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 0.58)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: right, reward: 1.79577662685
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.7957766268453457, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.80)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 2.53842223762
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 2.5384222376239363, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.54)
35% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 263
\-------------------------
Environment.reset(): Trial set up with start = (3, 4), destination = (8, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0721; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0721; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 2.35779116024
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.357791160240193, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 2.36)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 1.58513013744
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.5851301374388622, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.59)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.45559005659
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.4555900565923137, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.46)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.09952263652
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.0995226365170265, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.10)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.04474588477
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.0447458847742057, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.04)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 2.76459458483
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.7645945848303297, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.76)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 1.6012130242
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.601213024197627, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.60)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 1.38897309095
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.3889730909503928, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.39)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 0.959212996801
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 8, 'action': None, 'reward': 0.9592129968011143, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.96)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 0.826845006887
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.8268450068872941, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.83)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: right, reward: 1.19556621128
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.1955662112765697, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.20)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: forward, reward: 0.689767325491
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 0.6897673254913973, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'left')
Agent drove forward instead of right. (rewarded 0.69)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 1.80584771766
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.8058477176640486, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 1.81)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 0.907042992704
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 0.907042992704457, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.91)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: forward, reward: -9.33917578762
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'left', 'right'), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': -9.339175787617984, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', 'right')
Agent attempted driving forward through a red light. (rewarded -9.34)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.65211771165
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.6521177116478893, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.65)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 0.693725390522
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'right'), 'deadline': 9, 't': 16, 'action': None, 'reward': 0.6937253905215413, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 0.69)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: left, reward: 0.406767681668
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'right', None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 0.4067676816675416, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'right', None)
Agent drove left instead of right. (rewarded 0.41)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 0.917264807912
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 0.9172648079117462, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.92)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: right, reward: 2.20817562463
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'forward'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 2.208175624626443, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'forward')
Agent followed the waypoint right. (rewarded 2.21)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: 0.889339566243
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 5, 't': 20, 'action': 'forward', 'reward': 0.8893395662430723, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 0.89)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 2.29407436849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 21, 'action': None, 'reward': 2.29407436849202, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.29)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: -5.49900760089
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 3, 't': 22, 'action': None, 'reward': -5.499007600888804, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.50)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: -0.384849968271
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 2, 't': 23, 'action': 'right', 'reward': -0.3848499682711173, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove right instead of forward. (rewarded -0.38)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.37991502648
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.37991502647755, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.38)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 264
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (8, 7), deadline = 25
Simulating trial. . .
epsilon = 0.0714; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0714; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0714; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0714; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 1.49191429536
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.4919142953643856, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', 'left')
Agent properly idled at a red light. (rewarded 1.49)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: 1.57887440727
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.578874407272421, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.58)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: right, reward: 1.74336234199
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.743362341989492, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.74)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 1.86875003741
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.8687500374073025, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.87)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 2.91642030918
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.9164203091849483, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.92)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: 2.33904278528
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.33904278527672, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 2.34)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: left, reward: 1.73543537239
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.735435372388255, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.74)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: forward, reward: 2.33714941876
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 2.3371494187577637, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.34)
68% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 265
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (4, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0707; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: 2.32495012898
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.32495012897608, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.32)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: 1.74332923143
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.7433292314325202, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.74)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: 2.33577257147
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.3357725714712254, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.34)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: 1.64542751775
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.6454275177536268, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.65)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: 1.57717395997
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.5771739599723669, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.58)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: 1.85335914145
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.8533591414465553, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.85)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 2.40103754573
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.401037545726439, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.40)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 2.32469676756
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.3246967675559693, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.32)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: 0.191697645095
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.19169764509537035, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.19)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 2.25103625159
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.251036251593769, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.25)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: left, reward: 1.40106576162
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.4010657616218323, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.40)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 1.23272833068
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.2327283306782073, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.23)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 2.65790462089
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.6579046208911117, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.66)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: 2.11102739409
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 2.111027394088425, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.11)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 266
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (4, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0699; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0699; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0699; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0699; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: left, reward: 2.54219907794
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.5421990779408916, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 2.54)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.81111011945
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.811110119449485, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.81)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.92427042243
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.9242704224328104, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.92)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.63201650395
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.632016503951262, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.63)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 1.34185284413
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.3418528441280875, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.34)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.19568136038
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.1956813603769447, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.20)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 1.76011006822
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.7601100682153252, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.76)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 1.3183743209
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.3183743208988843, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.32)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.2575811002
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.2575811001989434, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.26)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: 1.71369328686
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.7136932868643022, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', None)
Agent drove forward instead of left. (rewarded 1.71)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 0.0886523291231
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'right'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.08865232912313925, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'right')
Agent drove right instead of left. (rewarded 0.09)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.1672433692
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'forward'), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.1672433691974498, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.17)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.91066061204
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.910660612043504, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.91)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 1.22438793079
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.2243879307910073, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.22)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: forward, reward: 0.485215742444
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 0.4852157424443836, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.49)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: forward, reward: -10.2505321176
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -10.250532117610955, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.25)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 1.36498353883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.3649835388267144, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.36)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 1.42913645876
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.4291364587572362, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.43)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: left, reward: -10.6612818397
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -10.661281839748838, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.66)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: left, reward: 0.628837954957
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 0.6288379549574031, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove left instead of right. (rewarded 0.63)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 267
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (6, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0693; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 1.72441775024
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.7244177502356433, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.72)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 1.8819708635
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.8819708634984076, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.88)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 2.86805117703
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.8680511770288555, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.87)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: right, reward: 1.4936133704
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.4936133704030174, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.49)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 1.34776710833
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.3477671083316152, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.35)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: left, reward: 2.1641832928
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.1641832928019102, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.16)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: right, reward: 0.995079435696
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.9950794356960864, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.00)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: right, reward: 0.365156095368
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.3651560953679953, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.37)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.04848670071
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.048486700712653, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.05)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: forward, reward: 0.0845930227908
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.0845930227908187, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.08)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: forward, reward: -9.95580990559
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': -9.955809905586463, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.96)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: 1.4494614971
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 1.4494614970993862, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.45)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.14434624263
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.144346242633475, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.14)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: left, reward: 1.97956555192
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.9795655519187625, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.98)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 1.4751637694
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.475163769403638, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.48)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 1.73536292494
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.7353629249363056, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.74)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: 1.56700615162
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.5670061516241982, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.57)
15% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 268
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (2, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0686; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0686; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0686; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: forward, reward: 1.70336781689
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.7033678168942201, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.70)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: None, reward: 1.16194591854
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.1619459185414847, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.16)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: None, reward: 1.60182767368
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.6018276736763335, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.60)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: 1.93051320033
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.9305132003258716, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.93)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: forward, reward: 1.78802221644
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.7880222164370925, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.79)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 2.23939269994
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.239392699938485, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.24)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 2.28081622891
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.2808162289106084, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.28)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 2.47659111521
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.4765911152067384, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.48)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: right, reward: 1.00774354994
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.0077435499431564, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.01)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 269
\-------------------------
Environment.reset(): Trial set up with start = (7, 2), destination = (3, 5), deadline = 35
Simulating trial. . .
epsilon = 0.0679; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0679; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0679; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 2.65243542442
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 35, 't': 0, 'action': None, 'reward': 2.652435424423791, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.65)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.33645492962
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 34, 't': 1, 'action': None, 'reward': 1.336454929619933, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.34)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 2.7267640004
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 33, 't': 2, 'action': None, 'reward': 2.7267640003979943, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.73)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: left, reward: 0.670521751659
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 32, 't': 3, 'action': 'left', 'reward': 0.670521751659447, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.67)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: forward, reward: 0.769153595733
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 31, 't': 4, 'action': 'forward', 'reward': 0.7691535957327951, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.77)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 1.87987132859
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 30, 't': 5, 'action': 'right', 'reward': 1.8798713285929887, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.88)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 2.60798802203
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 29, 't': 6, 'action': 'forward', 'reward': 2.607988022027265, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.61)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 1.87023840191
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 28, 't': 7, 'action': 'forward', 'reward': 1.8702384019054494, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.87)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 2.110036642
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 27, 't': 8, 'action': None, 'reward': 2.1100366419953644, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.11)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 2.12535137844
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 26, 't': 9, 'action': 'forward', 'reward': 2.125351378442483, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.13)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: 0.908364610138
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 25, 't': 10, 'action': 'left', 'reward': 0.9083646101383063, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 0.91)
69% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 270
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (5, 6), deadline = 30
Simulating trial. . .
epsilon = 0.0672; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 2.33562086614
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.3356208661411313, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.34)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 2.20991544557
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.209915445570572, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.21)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: -39.8809911847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', 'forward'), 'deadline': 28, 't': 2, 'action': 'left', 'reward': -39.88099118468482, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.88)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: forward, reward: 1.12560513838
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 1.1256051383781918, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.13)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: 1.74390276657
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 1.7439027665673008, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.74)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 1.55963709918
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.5596370991758102, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.56)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 0.807532425315
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 0.8075324253145414, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.81)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 0.247761150862
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 0.2477611508616916, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 0.25)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 1.35479685346
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'left'), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': 1.354796853464289, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'left')
Agent drove forward instead of right. (rewarded 1.35)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.57847758038
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.5784775803754034, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.58)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 2.12788010975
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 2.1278801097453948, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.13)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 2.7021279939
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 2.7021279938983875, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.70)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 2.04202175686
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': 2.0420217568557293, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.04)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 1.96652347349
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 17, 't': 13, 'action': None, 'reward': 1.9665234734860801, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.97)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 1.95302062552
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.9530206255188347, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.95)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.73831568884
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 15, 't': 15, 'action': None, 'reward': 2.738315688842036, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.74)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 2.53797588263
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': 2.537975882628377, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.54)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.51003291992
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 13, 't': 17, 'action': None, 'reward': 2.510032919915942, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.51)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.60012046498
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.6001204649811862, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.60)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.57010132686
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 11, 't': 19, 'action': None, 'reward': 2.5701013268565256, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.57)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.90596338414
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 20, 'action': None, 'reward': 1.905963384140622, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.91)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: -0.183574092467
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 9, 't': 21, 'action': 'right', 'reward': -0.18357409246735623, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded -0.18)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: left, reward: 0.708196308983
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 8, 't': 22, 'action': 'left', 'reward': 0.7081963089834131, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 0.71)
23% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 271
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (1, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0665; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0665; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0665; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: left, reward: -19.5461587649
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'left', 'right', 'forward'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -19.54615876494183, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.55)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 1.58903522056
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.5890352205649518, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.59)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 2.86550998447
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 2.865509984466525, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.87)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 2.53685982668
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.5368598266786773, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.54)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: left, reward: 2.53688928025
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 2.536889280251243, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 2.54)
75% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 272
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (5, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0659; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 2.53502255822
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.53502255821937, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.54)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 1.79669491779
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.7966949177862437, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.80)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 1.51196303578
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.5119630357771292, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.51)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 1.88643167026
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.8864316702642894, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.89)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 1.62059523094
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.6205952309410399, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.62)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: 1.98444178496
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.9844417849649478, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.98)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 1.24090767955
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.2409076795519172, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.24)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 2.58238109623
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.582381096230251, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.58)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 2.3324799484
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.3324799483996452, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.33)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: right, reward: 0.160343954267
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.1603439542670766, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove right instead of forward. (rewarded 0.16)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: 0.725601242188
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.7256012421879049, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 0.73)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: -9.33499718092
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': -9.334997180923486, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -9.33)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: left, reward: -9.82283388791
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 13, 't': 12, 'action': 'left', 'reward': -9.822833887911717, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.82)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 0.830118371117
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 12, 't': 13, 'action': None, 'reward': 0.8301183711169802, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.83)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 2.51688275869
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.5168827586852442, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.52)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 2.53524142506
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.5352414250641395, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.54)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: 1.52801209683
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'left'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 1.5280120968336224, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'left')
Agent drove forward instead of left. (rewarded 1.53)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 0.944443926354
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 8, 't': 17, 'action': None, 'reward': 0.9444439263544555, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.94)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 0.612353320823
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 7, 't': 18, 'action': None, 'reward': 0.6123533208225287, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 0.61)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.35665052562
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 6, 't': 19, 'action': None, 'reward': 2.3566505256165717, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.36)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 0.831804994247
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.8318049942470109, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.83)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: left, reward: 2.02904730002
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 2.029047300023473, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.03)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: 1.29671635234
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 3, 't': 22, 'action': 'left', 'reward': 1.2967163523365728, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.30)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 0.52123780454
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 2, 't': 23, 'action': None, 'reward': 0.5212378045399315, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.52)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.3899161017
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.389916101696562, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.39)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 273
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (2, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0652; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0652; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 1.25368095109
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.253680951087445, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.25)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.06428280429
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.0642828042923622, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.06)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.09291393659
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.0929139365894662, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.09)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 2.30897173967
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.3089717396657083, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.31)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 0.467445131589
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 0.46744513158938517, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent drove right instead of left. (rewarded 0.47)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 0.986376507671
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 0.9863765076714544, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 0.99)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 1.68428608694
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.684286086941266, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.68)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.72216704132
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.7221670413176655, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.72)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 0.981502854303
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 0.9815028543027899, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.98)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.46020510591
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.4602051059056156, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.46)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 1.28022111832
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.2802211183178973, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 1.28)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 1.43736615326
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.43736615325974, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.44)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 1.33372797173
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 13, 't': 12, 'action': 'left', 'reward': 1.3337279717312713, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 1.33)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 2.57416432566
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 12, 't': 13, 'action': None, 'reward': 2.574164325656522, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.57)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.10445000674
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.1044500067353884, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.10)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 0.621133299916
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 0.6211332999155046, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.62)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.09547984441
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.0954798444133234, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.10)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.0113867876
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 8, 't': 17, 'action': None, 'reward': 1.011386787596185, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.01)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 0.989836640196
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 0.9898366401963776, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 0.99)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.5591299198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 6, 't': 19, 'action': None, 'reward': 1.5591299197954085, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 1.56)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.56890077398
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.5689007739801857, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.57)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: left, reward: 1.94745427285
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 1.947454272847454, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.95)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 0.389387415669
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'right'), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.3893874156685333, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'right')
Agent drove right instead of left. (rewarded 0.39)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: left, reward: 0.671455144457
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 23, 'action': 'left', 'reward': 0.6714551444569514, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.67)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: left, reward: 0.798368316894
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 1, 't': 24, 'action': 'left', 'reward': 0.798368316894347, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 0.80)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 274
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (4, 2), deadline = 30
Simulating trial. . .
epsilon = 0.0646; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 2.86779098408
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 2.8677909840821174, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.87)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 1.05334987783
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.0533498778308894, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.05)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 2.72212837387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 2.722128373866237, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.72)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.4002297751
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.4002297750957455, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.40)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 2.16389930049
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 2.163899300488657, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.16)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 1.59322724238
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'right'), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.593227242382948, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'right')
Agent drove right instead of forward. (rewarded 1.59)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: forward, reward: 0.781884940178
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 0.7818849401778424, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded 0.78)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.68363830937
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.6836383093679088, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.68)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 2.19013473733
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.1901347373333215, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.19)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: left, reward: 1.80048644852
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 1.8004864485210315, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.80)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 2.59769676369
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 2.5976967636907373, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.60)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 1.49805966814
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.4980596681417488, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.50)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 1.55968901464
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 1.5596890146438656, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.56)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: left, reward: 0.617315570378
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 0.6173155703782436, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.62)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 1.08915305303
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.0891530530256675, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.09)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 1.63208052721
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.6320805272077943, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.63)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 2.65982613258
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 2.6598261325793153, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 2.66)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 1.85714256716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.8571425671635968, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.86)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 1.73647313713
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.7364731371260675, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.74)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: right, reward: 1.51128137046
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 1.511281370456785, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.51)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: left, reward: 2.11645983646
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 10, 't': 20, 'action': 'left', 'reward': 2.116459836457061, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 2.12)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 275
\-------------------------
Environment.reset(): Trial set up with start = (2, 2), destination = (5, 5), deadline = 30
Simulating trial. . .
epsilon = 0.0639; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: left, reward: 2.0772021914
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 2.0772021914020677, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.08)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.34961023615
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.3496102361492797, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.35)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 0.426190209861
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 0.42619020986064193, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.43)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: forward, reward: 1.80560912537
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 1.8056091253665816, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 1.81)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 0.85817387784
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 0.8581738778403846, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove right instead of left. (rewarded 0.86)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: left, reward: 2.57510589482
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 2.575105894820378, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.58)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: left, reward: 1.70267980673
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 1.7026798067299518, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.70)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: left, reward: 1.74252862435
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 1.7425286243528553, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 1.74)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: forward, reward: 0.749862685023
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': 0.7498626850234474, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.75)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 0.999777151395
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 0.9997771513945977, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.00)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 0.978078042357
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 20, 't': 10, 'action': None, 'reward': 0.9780780423566826, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 0.98)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 2.48551528953
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.485515289530869, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.49)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 0.971708584659
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 0.9717085846594171, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', 'left')
Agent properly idled at a red light. (rewarded 0.97)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: 2.67950688968
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 2.6795068896793, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.68)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: right, reward: 2.71650237453
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 2.7165023745319115, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.72)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: right, reward: 0.719740096985
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'right'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 0.7197400969846685, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'right')
Agent drove right instead of forward. (rewarded 0.72)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 0.951919598608
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 0.9519195986083628, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 0.95)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: right, reward: 1.19679459048
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 1.19679459048019, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.20)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: right, reward: 1.74420433883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'right'), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 1.7442043388274757, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'right')
Agent followed the waypoint right. (rewarded 1.74)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: forward, reward: 1.33836732953
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': 1.3383673295344278, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.34)
33% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 276
\-------------------------
Environment.reset(): Trial set up with start = (4, 3), destination = (8, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0633; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: -39.7852916648
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -39.7852916648234, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.79)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: left, reward: -10.3989863171
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -10.398986317076224, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.40)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: -10.5685133701
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': -10.568513370131118, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.57)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 2.7694760351
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.769476035099597, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.77)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 2.46935027565
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.4693502756534365, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.47)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 1.22705185284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.2270518528432506, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 1.23)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: left, reward: 1.80530107773
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.8053010777320762, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.81)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 2.06490191608
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.0649019160775524, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.06)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 1.88947726578
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.8894772657764027, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.89)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 1.96294737805
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.9629473780523432, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent followed the waypoint forward. (rewarded 1.96)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 2.79235925816
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.7923592581638363, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.79)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 2.19823248867
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.1982324886731757, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.20)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 1.68251694624
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.6825169462367455, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.68)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: left, reward: 1.77726333742
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 1.7772633374195037, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.78)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 2.23263761595
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 2.2326376159458086, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.23)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 1.58810545728
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.5881054572839106, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.59)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 1.57171921877
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.5717192187707776, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.57)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: forward, reward: 0.968834124909
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 0.9688341249093753, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 0.97)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 2.24648222308
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 7, 't': 18, 'action': None, 'reward': 2.24648222308409, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.25)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 1.50117428957
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 19, 'action': None, 'reward': 1.501174289568317, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.50)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 0.728537789344
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.7285377893441962, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.73)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: forward, reward: 0.980138680011
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 0.9801386800113412, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.98)
12% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 277
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (4, 6), deadline = 30
Simulating trial. . .
epsilon = 0.0627; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 2.43806273841
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.4380627384139757, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.44)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 1.82156514377
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.8215651437727758, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.82)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 1.81232307468
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.8123230746794403, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.81)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 0.388732720221
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 0.3887327202212646, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.39)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 1.10136928654
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 1.101369286543414, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.10)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 2.12319196934
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.123191969338624, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.12)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 1.37254334613
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.3725433461320886, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.37)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 2.77360250961
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 23, 't': 7, 'action': None, 'reward': 2.7736025096115036, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.77)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: forward, reward: -10.9175081102
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'right', 'forward'), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': -10.917508110213396, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', 'forward')
Agent attempted driving forward through a red light. (rewarded -10.92)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: forward, reward: 0.652479226285
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'forward'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 0.6524792262845001, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.65)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 1.16395309268
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.1639530926785366, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 1.16)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: 2.09024357294
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 2.0902435729415334, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.09)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 0.899876302736
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'forward'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 0.8998763027356226, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.90)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: None, reward: 2.05987569745
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 17, 't': 13, 'action': None, 'reward': 2.0598756974525463, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.06)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: None, reward: 1.46739112656
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.4673911265591586, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.47)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: None, reward: 2.11967653331
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 15, 't': 15, 'action': None, 'reward': 2.1196765333061824, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.12)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: None, reward: 2.17702555032
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 14, 't': 16, 'action': None, 'reward': 2.17702555032382, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.18)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: left, reward: 0.924020780189
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 13, 't': 17, 'action': 'left', 'reward': 0.9240207801885068, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.92)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 0.92084857422
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 18, 'action': None, 'reward': 0.9208485742198176, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.92)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 1.59449698526
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': 1.594496985261964, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 1.59)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 1.11393525758
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 1.1139352575820418, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.11)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 278
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (1, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0620; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 2.92643384472
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.9264338447246105, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.93)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.94649433409
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.946494334088128, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.95)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.04373533323
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.043735333229818, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.04)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 1.45724249478
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.457242494777931, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.46)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 2.46068613652
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.4606861365248553, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.46)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: 1.53106342203
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.5310634220337218, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.53)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.70289923357
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.702899233570076, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.70)
65% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 279
\-------------------------
Environment.reset(): Trial set up with start = (7, 2), destination = (1, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0614; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0614; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0614; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.77204987309
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.772049873090097, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.77)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 2.81101400727
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.8110140072695975, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.81)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.02275795728
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.0227579572764633, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.02)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: 1.59879403022
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.5987940302212338, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.60)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.83204199296
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.8320419929585925, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.83)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.39211395805
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.392113958049501, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.39)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.01281687892
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.012816878920241, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.01)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 2.76516270335
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.765162703351418, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.77)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: left, reward: 0.0844129253694
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'right', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 0.08441292536943801, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'right', None)
Agent drove left instead of right. (rewarded 0.08)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 2.26656057107
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 2.266560571068247, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.27)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 1.35480161151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.3548016115098114, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.35)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 1.60482490057
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.6048249005705375, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.60)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: 2.36912035086
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 2.3691203508634127, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 2.37)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 1.6449979663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.644997966295909, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.64)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: -0.261503033163
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'left'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': -0.2615030331627374, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'left')
Agent drove right instead of left. (rewarded -0.26)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 2.43888564226
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 2.43888564225696, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 2.44)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 1.36682911556
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', 'forward'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.366829115561964, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', 'forward')
Agent followed the waypoint right. (rewarded 1.37)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: -19.0759613064
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': -19.07596130640181, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.08)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: 0.944524930398
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 0.9445249303980512, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.94)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: left, reward: 1.80687758995
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 1.8068775899465768, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.81)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 280
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (8, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0608; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: 0.711912506753
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 0.7119125067532011, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 0.71)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: -10.757509538
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'forward'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -10.757509538020418, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -10.76)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 2.20413132268
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.2041313226761705, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.20)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 2.29321167775
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.293211677750231, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.29)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 1.35409807811
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.3540980781114003, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.35)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 2.7086599883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.708659988304305, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.71)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: 0.920214331654
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 0.9202143316538975, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 0.92)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 2.03016861004
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 2.030168610040402, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.03)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.34189519634
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.341895196335477, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.34)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 1.82437911788
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.8243791178806967, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.82)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 2.33063996258
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'right'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 2.3306399625753462, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 2.33)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 281
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (4, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0602; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 1.41544814529
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'right'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.4154481452905878, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'right')
Agent properly idled at a red light. (rewarded 1.42)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.07617292469
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.0761729246899976, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.08)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.88446479317
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.884464793174917, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.88)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: left, reward: 2.31878112044
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 2.3187811204422193, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.32)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 1.82045675361
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.8204567536064182, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.82)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 1.0721328461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.0721328460969035, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.07)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 1.63972444445
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.639724444452997, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.64)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: forward, reward: 1.30605062693
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.3060506269293943, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove forward instead of left. (rewarded 1.31)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 2.18984258204
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.189842582043309, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.19)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: forward, reward: 0.993517567031
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.9935175670309171, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.99)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: -5.7227844989
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': -5.722784498900191, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.72)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 1.87455342276
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.874553422763019, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.87)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 2.04558096665
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.0455809666481732, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.05)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: 1.84727778968
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.8472777896817583, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.85)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 1.53734638528
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.5373463852826914, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.54)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 1.91663752148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.9166375214833247, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.92)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 1.62599209432
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.6259920943232775, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.63)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: forward, reward: 0.707235643486
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.7072356434861568, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 0.71)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: forward, reward: -10.7940099539
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': -10.79400995385521, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.79)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 0.419034612185
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.41903461218475546, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.42)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 282
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (8, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0596; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0596; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0596; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0596; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 1.10223046945
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.1022304694460576, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.10)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: right, reward: 1.19753272521
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.1975327252134034, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent drove right instead of left. (rewarded 1.20)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: forward, reward: 1.24411352821
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 1.244113528207133, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.24)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: right, reward: 2.81457114066
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 2.8145711406558114, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.81)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 0.500084217879
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 0.5000842178790862, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.50)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 1.79423840073
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.794238400733839, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.79)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: 1.64490087028
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.6449008702848067, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.64)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 2.19861700119
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.1986170011867685, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.20)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: -39.306259277
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': -39.30625927696588, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.31)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: 0.890726943841
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 0.8907269438406227, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.89)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: 0.925863010534
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 0.9258630105335768, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.93)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.34036075422
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'forward'), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.340360754217447, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.34)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.42264411235
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.4226441123526445, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.42)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.63872839666
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.6387283966605017, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.64)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.57869476773
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'right'), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.5786947677261907, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'right')
Agent properly idled at a red light. (rewarded 2.58)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.99782048172
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.9978204817223606, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.00)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: forward, reward: 1.315081149
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 1.315081149002981, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.32)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: 0.758793086195
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 0.7587930861954009, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.76)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 1.51817467399
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 1.5181746739933892, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.52)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 2.11765522317
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'left'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 2.1176552231735823, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'left')
Agent followed the waypoint right. (rewarded 2.12)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 283
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (3, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0590; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0590; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0590; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0590; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: right, reward: 0.0147163325351
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.014716332535064769, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.01)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 1.49610505886
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.496105058857686, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.50)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 2.09201442697
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.09201442697317, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.09)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 2.11972814102
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.119728141018662, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.12)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: left, reward: 1.19518917845
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.1951891784461146, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.20)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 2.09853914711
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.09853914711087, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.10)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 1.57118349505
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.5711834950499088, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.57)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: forward, reward: 1.73273982706
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.7327398270575056, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 1.73)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 284
\-------------------------
Environment.reset(): Trial set up with start = (5, 7), destination = (3, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0584; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 2.75685742051
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.756857420506429, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.76)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 1.80494807993
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.8049480799328805, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.80)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 1.83548138761
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.835481387607145, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.84)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 1.2155269011
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.215526901097556, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.22)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: right, reward: 1.88454684563
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.8845468456343262, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.88)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: right, reward: 1.00320854785
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.0032085478479411, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.00)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 2.42563380641
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.425633806411436, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.43)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: forward, reward: 1.37155133908
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.3715513390847485, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.37)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 1.37238479937
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.3723847993686102, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.37)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 0.182828053622
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.18282805362172105, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.18)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 2.39770135936
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.3977013593553673, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.40)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 1.17421613876
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.1742161387575665, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.17)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 2.20187939366
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.201879393664652, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.20)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: left, reward: 1.57073279453
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.5707327945301803, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.57)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: forward, reward: 1.37115384846
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 1.371153848464287, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 1.37)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 1.00449601302
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.0044960130226523, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.00)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: left, reward: 0.598105000044
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 0.5981050000441515, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 0.60)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 0.735340917182
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.7353409171820153, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.74)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 1.28949751223
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.2894975122277479, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.29)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: left, reward: 0.755291949167
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 0.7552919491667662, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.76)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 285
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (4, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0578; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 1.16501943904
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.1650194390384914, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', None)
Agent drove right instead of forward. (rewarded 1.17)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 2.7369052531
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.736905253101405, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.74)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 2.91194933212
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.91194933211698, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.91)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 2.8404383672
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.8404383671985296, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.84)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.96145064229
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.9614506422911255, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.96)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: left, reward: 2.64607540243
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.646075402428113, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.65)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 2.6942315711
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.6942315711038614, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.69)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 2.01976032729
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.0197603272869156, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.02)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 2.78051949661
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.78051949660995, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.78)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.99971441394
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.9997144139403082, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.00)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 0.874519200498
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 0.8745192004982643, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.87)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 2.65816306305
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 2.658163063051944, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.66)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 1.06117625698
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.0611762569828285, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.06)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 2.19098142444
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 2.19098142444481, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.19)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: 1.03410537153
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 1.0341053715290354, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 1.03)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 2.44466367053
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.444663670527241, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.44)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 0.644412312621
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.6444123126209929, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.64)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: forward, reward: 1.00441676386
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 1.0044167638578776, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.00)
10% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 286
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (1, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0573; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: left, reward: 1.00124946357
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.0012494635704132, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.00)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: 2.16707228605
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.16707228604571, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 2.17)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: 2.30922160855
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.3092216085529857, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.31)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: left, reward: 1.07497163804
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 1.0749716380362044, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.07)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 2.75759333477
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.7575933347746067, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.76)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 2.46446800789
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 2.464468007888982, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.46)
70% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 287
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (5, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0567; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0567; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0567; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.70025309989
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.700253099887925, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.70)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.97644569584
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.9764456958423877, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.98)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.3921607201
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.3921607200987425, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.39)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.16467286402
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.164672864020168, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.16)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: right, reward: 1.28252097097
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.2825209709732555, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.28)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 1.47138317622
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.4713831762165535, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.47)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: 1.52508790262
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.525087902623322, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.53)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 1.30994838689
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.30994838688504, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.31)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 1.97483529251
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.9748352925089794, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.97)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: forward, reward: 1.66300656348
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.6630065634770417, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', None)
Agent followed the waypoint forward. (rewarded 1.66)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 288
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (3, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0561; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: left, reward: -9.31523091735
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -9.315230917352201, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.32)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 2.49212466804
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.4921246680363724, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.49)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 1.19420853401
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.1942085340109942, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.19)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 1.77172728856
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.7717272885554531, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.77)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 1.02790395811
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.027903958105328, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.03)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 2.69164502305
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.691645023054572, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.69)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: right, reward: 1.4712131877
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.4712131877028818, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.47)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: None, reward: 2.61686540754
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.61686540753706, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.62)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: None, reward: 1.57044706726
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.5704470672624935, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.57)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: None, reward: 2.09053828082
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.0905382808164683, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.09)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: 0.570112947969
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 0.5701129479686474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.57)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 1.76124556865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.761245568648527, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.76)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: 1.55510214099
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.5551021409900774, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.56)
35% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 289
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (2, 3), deadline = 30
Simulating trial. . .
epsilon = 0.0556; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: 2.12554401953
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 2.1255440195268087, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.13)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 1.49638241234
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'right'), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.496382412335145, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.50)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 2.70899830499
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.708998304985398, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.71)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 2.55889722099
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.5588972209913488, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.56)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 0.984673824366
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 0.9846738243663928, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.98)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 2.75114541273
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.7511454127322037, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.75)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: 0.943547404504
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 0.9435474045038597, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.94)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: 1.45888822393
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 1.458888223930897, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.46)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 1.74501688387
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': 1.745016883871857, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.75)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.50532210515
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.5053221051509331, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.51)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 2.83190934414
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.8319093441381686, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.83)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 2.80134783268
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 2.8013478326814387, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.80)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 2.52889748364
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.528897483639083, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.53)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 2.39371270471
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'left'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 2.393712704706925, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'left')
Agent followed the waypoint right. (rewarded 2.39)
53% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 290
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (2, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0550; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.49040121908
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'right'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.4904012190842326, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'right')
Agent properly idled at a red light. (rewarded 1.49)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.21566113761
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.215661137606353, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.22)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.26497251764
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.264972517641758, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.26)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.02122063112
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.021220631124991, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.02)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.33506839374
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'right'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.335068393737835, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'right')
Agent properly idled at a red light. (rewarded 2.34)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: 1.33366627706
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.333666277055054, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded 1.33)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.21220614742
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.2122061474201997, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.21)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 1.27398722591
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.2739872259063247, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.27)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 1.82573998489
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.8257399848924254, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.83)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 0.29084131238
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.2908413123804243, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.29)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: 1.8408079951
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 1.8408079951011678, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.84)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.77884993574
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.7788499357445917, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.78)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.35187191278
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.351871912782642, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.35)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.87961236197
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.879612361969107, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.88)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.34464049601
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.344640496010297, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.34)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.42778165371
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.427781653708494, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.43)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: forward, reward: 1.64946826001
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 1.6494682600124257, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.65)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: 1.524545377
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 1.5245453769997834, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded 1.52)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: -9.25981911279
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': 'left', 'reward': -9.259819112788884, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.26)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: 0.518929362943
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 0.51892936294264, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.52)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 0.593251000086
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.5932510000860016, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 0.59)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 0.622603929621
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 0.622603929621033, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.62)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 0.364016860863
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 3, 't': 22, 'action': None, 'reward': 0.3640168608629464, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.36)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.07929282779
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.079292827791341, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.08)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.32897405103
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.3289740510254204, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.33)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 291
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (5, 3), deadline = 30
Simulating trial. . .
epsilon = 0.0545; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: forward, reward: 1.84780266227
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'left', 'left'), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': 1.8478026622696164, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'left', 'left')
Agent drove forward instead of right. (rewarded 1.85)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 2.63180453543
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', 'forward'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 2.6318045354268795, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', 'forward')
Agent followed the waypoint right. (rewarded 2.63)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: left, reward: 1.32735663593
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 28, 't': 2, 'action': 'left', 'reward': 1.3273566359282918, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove left instead of forward. (rewarded 1.33)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 1.98140127928
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 1.9814012792833835, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.98)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 1.30598233999
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 1.3059823399853285, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 1.31)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: 2.79387588193
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 2.7938758819268243, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.79)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.31751673084
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.3175167308407978, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.32)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.26666608444
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.2666660844364537, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.27)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.34241415277
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.3424141527729767, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.34)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.59105278479
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 21, 't': 9, 'action': None, 'reward': 2.591052784794454, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.59)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.26114621991
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.2611462199051178, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.26)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: 2.16200377325
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'forward'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 2.162003773254939, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'forward')
Agent followed the waypoint forward. (rewarded 2.16)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 2.42533380031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.4253338003090557, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.43)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: 1.40871286499
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.4087128649880816, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded 1.41)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 2.03923626048
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 2.03923626047814, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.04)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 1.83183923421
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.8318392342050698, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.83)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: left, reward: 1.55496123809
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 1.5549612380898827, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.55)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 2.65210199965
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 13, 't': 17, 'action': None, 'reward': 2.652101999647556, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.65)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 2.34192803152
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 12, 't': 18, 'action': None, 'reward': 2.341928031524951, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.34)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 1.71993356079
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 11, 't': 19, 'action': None, 'reward': 1.7199335607906148, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.72)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 0.603572174032
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 20, 'action': None, 'reward': 0.6035721740318218, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.60)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: left, reward: 2.32795602911
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 21, 'action': 'left', 'reward': 2.3279560291063275, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.33)
27% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 292
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (4, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0539; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0539; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0539; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: left, reward: 1.51583090988
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.5158309098828937, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove left instead of right. (rewarded 1.52)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: left, reward: 1.04630019651
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.0463001965088987, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.05)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.21764029453
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.2176402945293907, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 1.22)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: forward, reward: 1.12777251952
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.1277725195153896, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.13)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.38047236525
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.3804723652514683, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.38)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.80823289845
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.8082328984536016, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.81)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: 2.50975574411
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.5097557441069682, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.51)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: 2.07660565714
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.0766056571404246, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.08)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 0.88045402231
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.8804540223103946, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.88)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: 1.86398721189
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.8639872118866463, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.86)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: forward, reward: 0.982665151226
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 0.9826651512262639, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.98)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: forward, reward: 0.976078029106
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.9760780291061528, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.98)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 1.7140367575
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.7140367574961364, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.71)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: -4.97300252724
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': -4.973002527241066, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.97)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: left, reward: 1.4090251351
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 1.4090251351034317, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.41)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: None, reward: 1.98921284153
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.989212841526529, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.99)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: None, reward: 0.754385430459
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.7543854304591522, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.75)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: left, reward: 1.73050917823
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 1.7305091782289967, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 1.73)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 1.22509393912
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.2250939391226798, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.23)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: -0.0648569538858
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 1, 't': 19, 'action': None, 'reward': -0.06485695388581669, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.06)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 293
\-------------------------
Environment.reset(): Trial set up with start = (5, 2), destination = (7, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0534; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 1.8794813866
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.8794813865960216, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.88)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 1.6431880961
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.6431880961033474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.64)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 1.9456356257
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.945635625700906, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.95)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 2.3495336439
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.349533643897407, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.35)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 1.09039413772
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.090394137722278, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.09)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.38288349596
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.38288349596266, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.38)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 1.68286341958
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.6828634195757346, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 1.68)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.62383090943
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.6238309094323549, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.62)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.25423956159
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.2542395615915969, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.25)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.70462250737
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.704622507368014, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.70)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: 0.899847122248
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 0.8998471222476343, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 0.90)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: right, reward: 1.26135645132
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.261356451324118, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.26)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 294
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (3, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0529; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0529; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0529; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.38212962797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.3821296279650308, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.38)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 2.12571868172
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.1257186817214127, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.13)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 2.34797115501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.347971155013419, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.35)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: left, reward: 1.66923643172
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 1.669236431720991, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.67)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.09045205211
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.0904520521105816, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.09)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: right, reward: 1.80772857886
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.8077285788612076, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.81)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: left, reward: 2.2293060512
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.2293060512026823, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.23)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.37554944825
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.3755494482528097, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.38)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 1.35760280046
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.3576028004609286, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 1.36)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: forward, reward: 0.870814070177
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.8708140701774567, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.87)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 2.70164345095
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.7016434509508724, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.70)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 2.60214763667
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'right'), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.602147636673941, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'right')
Agent properly idled at a red light. (rewarded 2.60)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.73643241887
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.7364324188715239, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.74)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: left, reward: 2.2812328316
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 2.281232831603306, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.28)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: 0.742375827391
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 0.7423758273910144, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 0.74)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: -20.6384697593
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'forward'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': -20.638469759321865, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.64)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 1.52552126231
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.5255212623107366, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.53)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: 0.325170925089
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'forward'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.3251709250892363, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 0.33)
10% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 295
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (4, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0523; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0523; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 1.68735707914
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'right', 'forward'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.687357079140654, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'right', 'forward')
Agent drove forward instead of left. (rewarded 1.69)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: forward, reward: 2.02747769217
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 2.0274776921727877, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent followed the waypoint forward. (rewarded 2.03)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: left, reward: 1.33968345631
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 1.3396834563079087, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', None)
Agent drove left instead of forward. (rewarded 1.34)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 1.88461425557
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.8846142555692857, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 1.88)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: left, reward: 1.71149724583
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 1.711497245828562, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.71)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: right, reward: 1.21195005545
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.2119500554543474, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.21)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 2.52373174992
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.5237317499225442, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.52)
72% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 296
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (6, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0518; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0518; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0518; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0518; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: left, reward: 0.709061823134
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 0.7090618231342798, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', None)
Agent drove left instead of right. (rewarded 0.71)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 1.06292280565
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.0629228056450786, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.06)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 1.96529689655
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.9652968965469575, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.97)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 1.81721978154
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.8172197815434181, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.82)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 2.4906185395
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.4906185394956726, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.49)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 0.975300402401
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.9753004024005597, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.98)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: -4.53970082841
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 19, 't': 6, 'action': None, 'reward': -4.539700828413656, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.54)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: left, reward: 2.66996257192
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 2.6699625719207405, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.67)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 2.72659226471
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.7265922647057215, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.73)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 1.14647499654
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.1464749965427006, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.15)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: forward, reward: 2.3368333984
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 2.3368333984008096, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.34)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 1.62804656414
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.6280465641383661, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.63)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 2.55677470331
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.5567747033117505, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.56)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 1.24421922772
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.24421922771959, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 1.24)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 0.986315438507
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 11, 't': 14, 'action': None, 'reward': 0.98631543850656, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.99)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 1.64186716572
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.641867165718035, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.64)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: left, reward: 1.79501925569
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 1.7950192556855307, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.80)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: left, reward: 1.47224084087
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 1.4722408408729, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 1.47)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 1.12161785648
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 1.1216178564811092, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.12)
24% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 297
\-------------------------
Environment.reset(): Trial set up with start = (4, 7), destination = (7, 4), deadline = 30
Simulating trial. . .
epsilon = 0.0513; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0513; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0513; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: left, reward: 2.82957843429
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 2.8295784342878143, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.83)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 2.8199335917
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.8199335917019726, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.82)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 2.93281603812
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.9328160381238995, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.93)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.06130634997
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.0613063499736273, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.06)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: left, reward: 1.17897723582
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 1.1789772358194317, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.18)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 2.4568872344
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 2.4568872343977706, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.46)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.81245094086
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.81245094086291, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.81)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.52766374204
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.527663742041878, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.53)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.41497053435
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.4149705343538335, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.41)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: left, reward: -10.0315746786
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': -10.031574678575153, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.03)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.61177105287
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.611771052866213, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.61)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: forward, reward: 1.74308501299
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.743085012988864, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.74)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: 2.59145473002
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 2.5914547300242665, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.59)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 2.63988233179
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 17, 't': 13, 'action': None, 'reward': 2.639882331793823, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.64)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 1.89137440663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.891374406632367, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.89)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 0.866774678443
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': None, 'reward': 0.8667746784434851, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.87)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 0.888226356625
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'right'), 'deadline': 14, 't': 16, 'action': None, 'reward': 0.8882263566248287, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'right')
Agent properly idled at a red light. (rewarded 0.89)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: forward, reward: 2.12935328265
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': 2.12935328265492, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.13)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 298
\-------------------------
Environment.reset(): Trial set up with start = (4, 7), destination = (8, 4), deadline = 35
Simulating trial. . .
epsilon = 0.0508; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0508; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 2.12406274989
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 35, 't': 0, 'action': None, 'reward': 2.1240627498865052, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.12)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 1.8830335301
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 34, 't': 1, 'action': None, 'reward': 1.8830335301040337, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.88)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 2.16535003509
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 33, 't': 2, 'action': None, 'reward': 2.165350035086113, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.17)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: left, reward: -9.44211176642
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 32, 't': 3, 'action': 'left', 'reward': -9.442111766420087, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.44)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 1.94147821447
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 31, 't': 4, 'action': None, 'reward': 1.941478214471031, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.94)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: left, reward: 2.08965283263
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 30, 't': 5, 'action': 'left', 'reward': 2.0896528326256307, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.09)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 1.99959839221
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 29, 't': 6, 'action': 'forward', 'reward': 1.9995983922125555, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.00)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.32014600456
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'left'), 'deadline': 28, 't': 7, 'action': None, 'reward': 1.3201460045555922, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.32)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 2.68975767439
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 27, 't': 8, 'action': 'forward', 'reward': 2.689757674394335, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.69)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: right, reward: 0.0956274155576
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 26, 't': 9, 'action': 'right', 'reward': 0.095627415557626, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.10)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: left, reward: 2.07340041738
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 10, 'action': 'left', 'reward': 2.0734004173832323, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.07)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.37901028802
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 24, 't': 11, 'action': None, 'reward': 1.379010288021703, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.38)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 2.13394055612
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 23, 't': 12, 'action': None, 'reward': 2.133940556123015, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.13)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 2.53339910082
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 22, 't': 13, 'action': 'right', 'reward': 2.533399100822881, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 2.53)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 0.403039949284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', 'forward'), 'deadline': 21, 't': 14, 'action': 'right', 'reward': 0.40303994928405873, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.40)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: 1.35106124462
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 20, 't': 15, 'action': 'left', 'reward': 1.3510612446173205, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.35)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 2.67465478973
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 19, 't': 16, 'action': None, 'reward': 2.6746547897340203, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.67)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 2.48066022641
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 18, 't': 17, 'action': None, 'reward': 2.480660226414414, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.48)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 1.32806935582
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 18, 'action': None, 'reward': 1.3280693558151972, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.33)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 0.897401013302
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 16, 't': 19, 'action': 'right', 'reward': 0.897401013302426, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 0.90)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 2.49677721194
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'right'), 'deadline': 15, 't': 20, 'action': 'right', 'reward': 2.4967772119390244, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'right')
Agent followed the waypoint right. (rewarded 2.50)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: 2.42285674972
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', None), 'deadline': 14, 't': 21, 'action': 'right', 'reward': 2.422856749718634, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', None)
Agent followed the waypoint right. (rewarded 2.42)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.13586846074
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 13, 't': 22, 'action': None, 'reward': 2.1358684607449483, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.14)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 0.870681481423
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 12, 't': 23, 'action': 'forward', 'reward': 0.870681481422561, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 0.87)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 1.5855479063
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 11, 't': 24, 'action': 'right', 'reward': 1.5855479062977982, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.59)
29% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 299
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (2, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0503; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0503; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 2.27368762085
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.273687620846154, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.27)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 1.0750433493
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.0750433492957272, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.08)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: right, reward: 2.918855422
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 2.918855422004582, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.92)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: 2.8735839094
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.873583909401322, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.87)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.51055868146
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.5105586814621077, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.51)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.72686487094
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.7268648709353658, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.73)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 2.7048769484
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.704876948396072, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 2.70)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 0.299379500081
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.2993795000810293, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 0.30)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: 2.63581607143
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 2.6358160714263494, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.64)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 2.55048740353
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 2.550487403525487, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.55)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: 1.71670540005
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'forward'), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.7167054000496151, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 1.72)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 1.52683214757
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.5268321475740134, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.53)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 2.36165263528
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.3616526352848592, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 2.36)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 0.0233360878635
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 12, 't': 13, 'action': None, 'reward': 0.02333608786350183, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.02)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: left, reward: 1.15948721852
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 1.1594872185239122, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.16)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 300
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (5, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0498; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 2.12460836683
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.1246083668331197, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.12)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 1.7713888851
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.7713888850993187, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.77)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 2.71843163303
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.718431633032904, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.72)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 2.48370166067
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.483701660669694, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.48)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 2.15063171096
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.150631710961606, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.15)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 1.04409725727
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.0440972572710672, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.04)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 0.615405832812
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'right', 'left'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.615405832812471, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'right', 'left')
Agent drove right instead of forward. (rewarded 0.62)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: left, reward: 2.79710733007
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 2.7971073300725005, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.80)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.32429110489
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.3242911048922776, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.32)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.08948754475
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.0894875447513082, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.09)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 0.800556725405
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 0.8005567254052344, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.80)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 1.79729208301
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.7972920830112278, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.80)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: forward, reward: 1.86195734334
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 1.8619573433405494, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.86)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: forward, reward: 1.17300456497
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.1730045649734517, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.17)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: None, reward: 1.11311689732
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.1131168973216117, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.11)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: left, reward: 0.938049332572
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 0.9380493325722303, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.94)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: forward, reward: 0.674243802746
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 0.674243802745772, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.67)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: 1.88947937114
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.8894793711406008, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.89)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: forward, reward: -40.4137328912
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', 'forward'), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': -40.41373289121928, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.41)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: forward, reward: -0.231951027325
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -0.23195102732453365, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove forward instead of left. (rewarded -0.23)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 301
\-------------------------
Environment.reset(): Trial set up with start = (3, 2), destination = (4, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0493; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 0.522913334173
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.5229133341726471, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.52)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 2.2857439611
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.285743961095569, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.29)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 2.41770122579
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.41770122578552, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.42)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 1.8972795148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.897279514802802, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.90)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: left, reward: 1.91586339138
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.915863391376796, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.92)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 2.29249700414
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.292497004136797, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.29)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 1.54147105729
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.5414710572933286, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.54)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 0.350752098761
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.3507520987607644, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.35)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 1.88943784619
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.8894378461890884, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.89)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: left, reward: 2.51991849976
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 2.5199184997614497, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.52)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 0.941873492275
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': 0.941873492275356, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 0.94)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 1.49528536075
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.495285360752294, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.50)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 1.48821237558
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'left'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 1.4882123755760785, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'left')
Agent drove forward instead of right. (rewarded 1.49)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 0.629235826129
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 0.6292358261286681, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.63)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 1.57169562635
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.5716956263513802, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.57)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 1.2673990422
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.2673990421962626, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.27)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 0.667706232102
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.6677062321022968, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.67)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 1.59627052418
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.5962705241778354, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.60)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: -0.292706359562
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': -0.2927063595623809, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded -0.29)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.90962819495
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.90962819495059, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.91)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 302
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (4, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0488; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 2.67735112667
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.6773511266694405, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.68)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 1.4101931643
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'right', None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.4101931643017909, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', None)
Agent followed the waypoint right. (rewarded 1.41)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 1.74548618768
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 1.7454861876788013, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.75)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 1.24732863831
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.2473286383061288, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent followed the waypoint forward. (rewarded 1.25)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 2.12187252146
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 2.1218725214582355, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.12)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 2.47872701636
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.478727016361643, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.48)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 2.20038386141
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.2003838614079685, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.20)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 1.95969254098
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.9596925409779813, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.96)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 0.0139931446792
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.013993144679186664, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.01)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: -10.4753826322
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': -10.475382632237368, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.48)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: -9.81881841054
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': -9.818818410535565, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.82)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 0.50826576714
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'right'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 0.5082657671398697, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'right')
Agent drove forward instead of left. (rewarded 0.51)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 2.02257969041
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.022579690412339, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.02)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.49601858145
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.4960185814540157, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.50)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.75047519584
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.7504751958411808, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 1.75)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 2.29076549893
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.2907654989329584, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 2.29)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: left, reward: 2.2171631996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 2.2171631995950536, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 2.22)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 0.695979886602
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 8, 't': 17, 'action': None, 'reward': 0.6959798866020528, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 0.70)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.84332346299
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.8433234629862727, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.84)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 2.0060088868
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 6, 't': 19, 'action': None, 'reward': 2.006008886804725, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.01)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 2.03384599605
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 2.033845996051086, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.03)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: left, reward: 1.45664372775
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 1.4566437277544462, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.46)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 1.32101578052
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': 1.3210157805238165, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.32)
8% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 303
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (3, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0483; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0483; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0483; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: forward, reward: -9.02769136967
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -9.027691369672928, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.03)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 1.29447786532
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.2944778653210132, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.29)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 2.36786421908
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.367864219080824, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.37)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: left, reward: 0.997897550218
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 0.9978975502178213, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.00)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 1.64053859615
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.6405385961534182, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.64)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 2.80509895099
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.805098950985985, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.81)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 2.47312548557
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.4731254855686218, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.47)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: 0.950292688392
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 0.9502926883922063, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 0.95)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 2.18827463754
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.1882746375408404, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.19)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 304
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (2, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0478; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0478; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0478; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0478; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0478; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0478; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0478; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.71361831766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.7136183176649058, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.71)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.39049495326
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.3904949532586677, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.39)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 2.72815023496
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.72815023495591, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.73)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 2.63205044705
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.632050447053482, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 2.63)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: forward, reward: -40.2871775685
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -40.28717756848232, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.29)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 2.50845401802
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.50845401801901, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.51)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: -0.0626552164663
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': -0.0626552164662525, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove right instead of left. (rewarded -0.06)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.2994636075
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.299463607498491, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.30)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.32217457325
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.322174573253773, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.32)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.76367225909
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.7636722590925158, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.76)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.45556528351
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.4555652835064217, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 1.46)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: left, reward: 1.59786578287
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.5978657828726524, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.60)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 2.32654754103
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.3265475410340217, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.33)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: 2.1305757421
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 2.130575742098495, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.13)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: 0.414027398888
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 0.4140273988882309, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 0.41)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.41499222738
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.4149922273826578, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.41)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: forward, reward: 0.195326913112
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 0.19532691311209482, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.20)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 2.23263858689
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 17, 'action': None, 'reward': 2.2326385868888297, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.23)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 1.10945748977
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.1094574897713463, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.11)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 1.15676502885
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 1.156765028845797, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 1.16)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 0.672684661201
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 0.6726846612008825, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.67)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 0.466645587077
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 0.46664558707727366, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.47)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 1.12504564379
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 1.125045643791389, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 1.13)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 1.23483237327
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.2348323732737765, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.23)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 0.778680422864
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.7786804228642892, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 0.78)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 305
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (8, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0474; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 1.83294789193
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.8329478919271578, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent followed the waypoint forward. (rewarded 1.83)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.93719584053
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.9371958405269194, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.94)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: -10.0314044375
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': -10.031404437493457, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.03)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 2.03544244617
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.0354424461715777, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.04)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.40888026271
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.408880262713216, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.41)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 1.98014449382
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.980144493816669, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.98)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 1.63647780051
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.6364778005135432, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.64)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: left, reward: 1.34773089016
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 1.34773089016061, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', None)
Agent drove left instead of forward. (rewarded 1.35)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 1.27238475452
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.2723847545220657, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.27)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.72460549023
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.7246054902277246, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.72)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: left, reward: 2.7955650115
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 2.795565011495674, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 2.80)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 306
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (7, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0469; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0469; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0469; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.95552085323
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.955520853225151, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.96)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.29200908787
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.292009087871613, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.29)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.60046550687
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'right'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.6004655068702767, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 2.60)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 0.983176807074
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 0.9831768070743618, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.98)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.27108258387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.2710825838714965, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.27)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 1.12825292411
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.1282529241148, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.13)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 1.16828135855
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.1682813585512257, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.17)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 2.09969254841
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.099692548412963, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.10)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.26978837718
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.2697883771752083, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.27)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 1.7673387189
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.7673387189034715, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.77)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: left, reward: 1.45452824872
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.4545282487230895, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.45)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 307
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (4, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0464; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: left, reward: 2.88059911258
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.880599112577122, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 2.88)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 2.71474653051
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.714746530509459, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.71)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 1.48393292733
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.4839329273342767, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.48)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 1.7930499188
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.7930499188001463, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.79)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.65663584436
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.6566358443597713, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.66)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 0.625262867448
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.6252628674480092, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove right instead of forward. (rewarded 0.63)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.17228720787
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.1722872078675515, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.17)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.83904515483
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.8390451548311475, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 2.84)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 0.935476095024
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 0.9354760950242744, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 0.94)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.44042477441
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.4404247744092267, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.44)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 0.933359433058
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 0.9333594330582076, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.93)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: left, reward: 1.54523567985
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 1.5452356798549958, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.55)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 1.40258269986
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.402582699859235, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.40)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: left, reward: 2.0273245177
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 2.0273245176975063, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.03)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: -5.09264475159
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': -5.092644751585311, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.09)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: right, reward: 2.39353513037
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'left'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 2.3935351303712014, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'left')
Agent followed the waypoint right. (rewarded 2.39)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 308
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (5, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0460; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0460; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0460; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0460; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 2.25314645992
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.2531464599217332, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.25)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 2.63439824545
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.6343982454487267, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.63)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 2.97173472272
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.9717347227186943, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.97)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 2.80622567677
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.806225676771893, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.81)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.94785157667
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.9478515766749713, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.95)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: 0.310085337963
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 0.31008533796264925, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.31)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: 2.33059652894
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.3305965289355526, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.33)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.39532061244
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.3953206124408697, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.40)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.23141855493
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.2314185549276964, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.23)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: 2.25650835781
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.2565083578055174, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.26)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.87884171034
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'right'), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.8788417103439825, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.88)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.75316653394
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.7531665339353473, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.75)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.54228419663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.542284196625605, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.54)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: 1.33620681693
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 1.3362068169314725, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.34)
44% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 309
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (5, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0455; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: 1.02665578071
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.0266557807128454, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.03)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 2.38159724796
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.381597247964315, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.38)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: -9.02304314796
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': -9.023043147960655, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.02)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 2.5451757188
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.5451757188009267, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.55)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: forward, reward: 2.09776847689
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.0977684768891907, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.10)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: right, reward: 2.08755623735
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 2.087556237345475, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.09)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: forward, reward: 1.97307268814
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.9730726881420682, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.97)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 2.45960709376
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'forward'), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.459607093755533, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.46)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 1.38039458111
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.3803945811068463, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.38)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 2.11140285475
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.1114028547457475, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.11)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: forward, reward: 2.50815633046
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 2.5081563304603964, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent followed the waypoint forward. (rewarded 2.51)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 310
\-------------------------
Environment.reset(): Trial set up with start = (8, 7), destination = (1, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0450; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0450; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 2.72978537352
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.729785373521053, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.73)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.67802921998
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.6780292199836737, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.68)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 2.48569081567
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.4856908156716635, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.49)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.45056853067
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.450568530669655, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.45)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.07208441166
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.0720844116570503, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.07)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.41551599273
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.4155159927257441, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.42)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: left, reward: 0.909377735362
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 0.909377735362024, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.91)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: left, reward: 2.5025970493
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 2.5025970493046836, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.50)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 1.24561714339
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.2456171433873304, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.25)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: forward, reward: 2.04859248823
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 2.0485924882270896, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.05)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 311
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (5, 7), deadline = 35
Simulating trial. . .
epsilon = 0.0446; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0446; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0446; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 1.59205085746
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 35, 't': 0, 'action': 'right', 'reward': 1.5920508574562082, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.59)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 2.81092207653
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 34, 't': 1, 'action': 'forward', 'reward': 2.810922076532896, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.81)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 0.136873812195
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'left'), 'deadline': 33, 't': 2, 'action': None, 'reward': 0.13687381219528172, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.14)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: 2.79072005989
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 32, 't': 3, 'action': 'forward', 'reward': 2.7907200598900728, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.79)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: 1.36326186816
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 31, 't': 4, 'action': 'right', 'reward': 1.3632618681550395, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.36)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: 1.42086161866
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 30, 't': 5, 'action': None, 'reward': 1.4208616186555307, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.42)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: 1.48476807958
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 29, 't': 6, 'action': None, 'reward': 1.4847680795765086, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.48)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: 2.76022422586
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 28, 't': 7, 'action': 'left', 'reward': 2.7602242258625527, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.76)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: right, reward: 2.34217608373
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 27, 't': 8, 'action': 'right', 'reward': 2.342176083734152, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.34)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: None, reward: 1.05908453061
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 26, 't': 9, 'action': None, 'reward': 1.0590845306136267, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.06)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: left, reward: 1.36690463181
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 25, 't': 10, 'action': 'left', 'reward': 1.3669046318101907, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.37)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 1.26954568167
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 24, 't': 11, 'action': 'right', 'reward': 1.2695456816670363, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.27)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 1.07559604239
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 23, 't': 12, 'action': None, 'reward': 1.0755960423879052, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.08)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: -0.131112658143
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 22, 't': 13, 'action': 'forward', 'reward': -0.13111265814293482, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent drove forward instead of right. (rewarded -0.13)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 0.850487930853
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 21, 't': 14, 'action': 'right', 'reward': 0.8504879308525652, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 0.85)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 1.25264460038
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 20, 't': 15, 'action': None, 'reward': 1.2526446003757918, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.25)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 2.6310098207
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 19, 't': 16, 'action': 'right', 'reward': 2.631009820696261, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.63)
51% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 312
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (1, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0442; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.29842990244
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.298429902435316, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.30)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.62708339935
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.627083399346044, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.63)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.53467242019
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.534672420186244, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.53)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 2.42995923926
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.4299592392598663, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.43)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.94402073658
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.944020736575652, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.94)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: 2.86109921091
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 2.8610992109114264, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.86)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: -40.0747663535
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -40.074766353500024, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.07)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.08699604902
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.0869960490158894, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.09)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.45300958956
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.453009589564565, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.45)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.19537299227
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.1953729922662872, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.20)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.35186670387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.3518667038736991, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.35)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 1.91507850676
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.9150785067604474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.92)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.95917078746
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.9591707874552655, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.96)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.14929737893
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.149297378927137, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.15)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.28565954724
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.2856595472362597, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.29)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 0.647837690911
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.6478376909105379, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.65)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: 1.72628602939
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.7262860293911508, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.73)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 0.56903622205
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.5690362220500351, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 0.57)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 1.486345316
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 1.4863453160006221, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.49)
5% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 313
\-------------------------
Environment.reset(): Trial set up with start = (3, 4), destination = (8, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0437; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 2.33685964697
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.3368596469712912, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', 'left')
Agent properly idled at a red light. (rewarded 2.34)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 2.51761860309
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.5176186030915826, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 2.52)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 2.12926043773
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 2.1292604377314164, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 2.13)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: right, reward: 0.90105672303
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'right'), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 0.901056723029636, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'right')
Agent drove right instead of forward. (rewarded 0.90)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 1.47276133852
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.4727613385186054, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.47)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 1.6376147046
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.637614704598634, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.64)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: 1.77130405972
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.7713040597196459, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.77)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 1.61263608412
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.6126360841218836, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.61)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 2.58846903175
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.588469031754495, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.59)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: left, reward: 2.63989185547
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 2.6398918554722117, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.64)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 1.75842339496
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 1.7584233949616985, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.76)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 1.76845564593
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.768455645933655, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.77)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 2.61154760423
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.6115476042256702, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.61)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 1.94823023398
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.9482302339779731, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.95)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 2.37722004606
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 2.3772200460556254, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.38)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.10371613199
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.1037161319873952, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.10)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: 2.37523261609
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 2.3752326160923714, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.38)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: -19.3622252989
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'right'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'right', 'right', 'forward'), 'deadline': 8, 't': 17, 'action': 'left', 'reward': -19.362225298938124, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'right', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.36)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: -0.0561111936807
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': -0.0561111936806733, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded -0.06)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 2.01363840881
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 19, 'action': None, 'reward': 2.013638408805659, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.01)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 1.37197040708
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.3719704070810237, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.37)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: left, reward: 0.781060167026
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 0.7810601670258519, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.78)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.01296381588
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 3, 't': 22, 'action': None, 'reward': 1.0129638158836423, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.01)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.24410853552
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.244108535523536, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.24)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.61681187255
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.6168118725457967, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.62)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 314
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (5, 7), deadline = 30
Simulating trial. . .
epsilon = 0.0433; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0433; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 2.46183274365
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.461832743653269, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.46)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 2.03223469855
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.0322346985457087, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.03)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 0.420771703981
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 0.42077170398098696, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent drove right instead of left. (rewarded 0.42)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 2.60446197403
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 2.6044619740325112, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.60)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 2.18692012514
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 2.186920125143575, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.19)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: 0.987242209827
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 0.9872422098274696, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent followed the waypoint forward. (rewarded 0.99)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: forward, reward: 2.41685081782
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 2.416850817815172, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.42)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.52473699958
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'right'), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.5247369995750737, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'right')
Agent properly idled at a red light. (rewarded 1.52)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 2.54611377104
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.5461137710358184, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.55)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.90356031453
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', None), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.9035603145343423, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 1.90)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: 1.15631462299
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.1563146229879637, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.16)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 0.186401080838
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 0.18640108083771822, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.19)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 2.35908417699
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.359084176987958, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.36)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: left, reward: 2.59813236935
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 2.5981323693451026, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.60)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 1.58189471251
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.58189471250681, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.58)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 0.95325241148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': None, 'reward': 0.9532524114798244, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.95)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: left, reward: 2.16746301837
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 2.1674630183733354, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.17)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 1.09157995349
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 1.0915799534941242, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.09)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 315
\-------------------------
Environment.reset(): Trial set up with start = (3, 2), destination = (7, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0429; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: -20.6920186573
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': -20.692018657335872, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.69)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 2.00266933727
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.0026693372728417, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.00)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 1.01901537487
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.0190153748686117, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.02)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: forward, reward: 0.0522856409102
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'forward'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 0.05228564091020049, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'forward')
Agent drove forward instead of left. (rewarded 0.05)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: 2.59909720055
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 2.599097200553537, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.60)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 1.50011202132
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.5001120213163341, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.50)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 1.2131192785
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.2131192784990346, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 1.21)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 1.64648002298
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.646480022983745, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.65)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.16042602592
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.160426025917249, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.16)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 1.67509805366
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.6750980536644158, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.68)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.62659235771
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.6265923577059733, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.63)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: left, reward: 1.26206758026
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 1.2620675802550412, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.26)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 316
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (4, 5), deadline = 35
Simulating trial. . .
epsilon = 0.0424; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0424; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: left, reward: 2.30254320015
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 35, 't': 0, 'action': 'left', 'reward': 2.3025432001481643, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.30)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.94001032866
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 34, 't': 1, 'action': None, 'reward': 1.9400103286613846, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.94)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.81895438454
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 33, 't': 2, 'action': None, 'reward': 1.8189543845425356, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.82)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 0.189029279445
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 32, 't': 3, 'action': 'right', 'reward': 0.18902927944481784, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.19)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 1.93592580262
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 31, 't': 4, 'action': None, 'reward': 1.935925802618309, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.94)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 2.61984098619
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 30, 't': 5, 'action': None, 'reward': 2.6198409861900003, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.62)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 2.31496928693
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 29, 't': 6, 'action': None, 'reward': 2.314969286927942, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 2.31)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 1.45874971855
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 28, 't': 7, 'action': 'right', 'reward': 1.4587497185497278, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.46)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 2.332965689
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 27, 't': 8, 'action': None, 'reward': 2.332965689001603, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.33)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: left, reward: 1.19004576713
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 26, 't': 9, 'action': 'left', 'reward': 1.1900457671291007, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.19)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.60722548131
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 25, 't': 10, 'action': None, 'reward': 1.60722548131119, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.61)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: forward, reward: 0.694245562544
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 24, 't': 11, 'action': 'forward', 'reward': 0.6942455625437235, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.69)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: forward, reward: 1.46280962828
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'right'), 'deadline': 23, 't': 12, 'action': 'forward', 'reward': 1.4628096282758594, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'right')
Agent drove forward instead of left. (rewarded 1.46)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 2.6977473337
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 22, 't': 13, 'action': None, 'reward': 2.697747333703356, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 2.70)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 1.43070192381
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 21, 't': 14, 'action': None, 'reward': 1.430701923808191, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 1.43)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 1.77432439093
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'left'), 'deadline': 20, 't': 15, 'action': None, 'reward': 1.7743243909312918, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.77)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: 2.71306012185
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 19, 't': 16, 'action': 'left', 'reward': 2.713060121850471, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.71)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: left, reward: 0.85180113852
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 18, 't': 17, 'action': 'left', 'reward': 0.8518011385197871, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.85)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 2.35080562619
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 17, 't': 18, 'action': 'right', 'reward': 2.3508056261877366, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.35)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.88147759
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 16, 't': 19, 'action': None, 'reward': 1.8814775900005745, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.88)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.68822992231
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 20, 'action': None, 'reward': 1.6882299223089812, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.69)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: forward, reward: 1.20781298902
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 21, 'action': 'forward', 'reward': 1.2078129890152192, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.21)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 0.653160418554
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 22, 'action': 'forward', 'reward': 0.6531604185538824, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.65)
34% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 317
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (5, 2), deadline = 30
Simulating trial. . .
epsilon = 0.0420; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: left, reward: 1.20137866366
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 1.2013786636560488, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.20)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: 1.46736124933
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 29, 't': 1, 'action': 'left', 'reward': 1.4673612493347588, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.47)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: forward, reward: 2.12340412964
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 2.1234041296381525, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.12)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: left, reward: -39.3452753726
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 27, 't': 3, 'action': 'left', 'reward': -39.34527537264039, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.35)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 0.00659084674507
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 0.006590846745071377, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.01)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.61866487285
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.6186648728523412, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.62)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 1.02890534391
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'forward'), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 1.028905343905107, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'forward')
Agent drove right instead of left. (rewarded 1.03)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: left, reward: 2.66815744931
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 2.6681574493139664, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.67)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: left, reward: 1.23175368182
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': 1.2317536818179005, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.23)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: right, reward: -0.0104887294764
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 21, 't': 9, 'action': 'right', 'reward': -0.010488729476362035, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded -0.01)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: left, reward: 1.74051167592
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 1.7405116759213057, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.74)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 2.66687791609
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.666877916085367, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.67)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 2.748435224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.748435224002046, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.75)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: left, reward: 2.63975341504
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 2.639753415043665, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.64)
53% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 318
\-------------------------
Environment.reset(): Trial set up with start = (5, 5), destination = (2, 2), deadline = 30
Simulating trial. . .
epsilon = 0.0416; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0416; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: forward, reward: 1.7236362465
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'right'), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': 1.723636246498049, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'right')
Agent drove forward instead of right. (rewarded 1.72)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: None, reward: 2.76370153018
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.7637015301775225, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.76)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: right, reward: 0.47389697048
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 0.47389697047951607, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.47)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: None, reward: 2.54318179675
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'forward'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.5431817967545998, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.54)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: None, reward: 2.34859601766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.348596017664784, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.35)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: forward, reward: 0.5808974451
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 0.580897445100009, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.58)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 2.32539485698
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.325394856981714, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.33)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 2.29066519035
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 2.290665190354404, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.29)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 1.66881945718
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.6688194571781063, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.67)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: left, reward: 2.03031084042
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 2.0303108404197205, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.03)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.85810677132
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.8581067713242543, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.86)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 1.2348573949
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.2348573949011572, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.23)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.54894659353
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.5489465935276938, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.55)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 2.32359829965
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 13, 'action': None, 'reward': 2.323598299654722, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.32)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.08316251483
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.083162514834085, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.08)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 2.7071958515
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 15, 'action': None, 'reward': 2.707195851504694, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.71)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: 1.51270649802
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': 1.5127064980214016, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.51)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 0.345531880946
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 0.345531880945835, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.35)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 1.97225121906
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.9722512190613517, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.97)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 0.527953265725
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'right'), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 0.5279532657246734, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'right')
Agent drove right instead of left. (rewarded 0.53)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 1.57007222401
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 1.5700722240122436, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.57)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 2.49428120203
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 2.4942812020279748, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.49)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 0.418191920827
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'forward'), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 0.4181919208265077, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.42)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: forward, reward: 0.864459766832
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 7, 't': 23, 'action': 'forward', 'reward': 0.8644597668320516, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.86)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 1.29118283153
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 1.2911828315257323, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent drove right instead of left. (rewarded 1.29)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 1.26571652146
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'forward', None), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 1.265716521455833, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'forward', None)
Agent followed the waypoint right. (rewarded 1.27)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 1.55699838334
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 4, 't': 26, 'action': 'right', 'reward': 1.556998383337992, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.56)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.20415454392
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 3, 't': 27, 'action': None, 'reward': 1.2041545439198154, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.20)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 0.625249947426
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 2, 't': 28, 'action': None, 'reward': 0.6252499474257023, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.63)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 1.8486912631
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 1, 't': 29, 'action': 'forward', 'reward': 1.8486912630958767, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 1.85)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 319
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (3, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0412; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.29032413267
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.290324132671683, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.29)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.26075666414
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.260756664139596, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.26)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.38210483874
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.382104838739379, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.38)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.46643165069
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.4664316506870394, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.47)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.49363272323
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.493632723233387, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.49)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.62908748202
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.6290874820173302, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.63)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 0.953925349383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 0.9539253493828428, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.95)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 1.40158093444
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.4015809344388948, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 1.40)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: forward, reward: -0.0522337408763
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -0.052233740876328816, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded -0.05)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: 1.37082527857
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.3708252785741954, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 1.37)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 2.77144556974
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 2.7714455697435527, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.77)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 2.58379565942
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 2.583795659421341, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 2.58)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: left, reward: 1.64274955284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 13, 't': 12, 'action': 'left', 'reward': 1.6427495528358584, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 1.64)
48% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 320
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (2, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0408; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0408; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 2.89177162253
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.891771622525875, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.89)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.61546544082
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.615465440821337, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.62)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.05490656255
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.0549065625478082, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.05)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.94243893407
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.9424389340705868, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.94)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 2.72174776421
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.7217477642130676, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.72)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 1.42800386847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.4280038684670293, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.43)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 2.8535163103
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.8535163102999967, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.85)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: right, reward: 1.7294191123
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.7294191123044775, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.73)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 2.04890877441
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.048908774409176, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.05)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: left, reward: 1.87543602399
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.8754360239856127, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.88)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 2.50806183991
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.5080618399059147, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.51)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: 2.05292542885
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 2.0529254288504095, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.05)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 321
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (6, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0404; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0404; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.18198150911
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.181981509107639, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.18)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: right, reward: 1.96472244313
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.964722443132955, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.96)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: 1.94025982004
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.9402598200446324, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 1.94)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: 1.79620719601
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.7962071960078538, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.80)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: 1.13553835604
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.135538356036366, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.14)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: 2.59606997774
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.5960699777437624, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.60)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: right, reward: 1.66033762793
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.660337627933074, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', None)
Agent drove right instead of forward. (rewarded 1.66)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: left, reward: 1.80046094938
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.8004609493835564, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.80)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 1.97709028515
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.9770902851544907, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.98)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: 1.64943596881
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.6494359688052804, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.65)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 322
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (7, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0400; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0400; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 1.09969183374
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.0996918337363653, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.10)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: 1.13809452616
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.1380945261583832, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 1.14)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 2.84830372103
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.8483037210329103, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.85)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 2.586889029
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.5868890289950146, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.59)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 2.43710474645
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.4371047464530076, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.44)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 2.16223869714
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.1622386971358063, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.16)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: 1.18208594296
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.182085942963791, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.18)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: 1.76377266699
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.763772666990542, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', 'left')
Agent drove forward instead of right. (rewarded 1.76)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.69086702
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.6908670199994162, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.69)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.75387831557
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.753878315571521, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.75)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: right, reward: 1.58231976429
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.5823197642886395, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.58)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 2.26160374714
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 2.2616037471412938, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.26)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 2.21692369743
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.216923697429908, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.22)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 2.36313305363
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 2.363133053628885, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.36)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 0.823089637742
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.8230896377422887, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 0.82)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: -9.85423075568
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -9.854230755678904, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.85)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 2.20830638142
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 4, 't': 16, 'action': None, 'reward': 2.2083063814160155, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 2.21)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: -0.541828233754
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -0.5418282337537849, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded -0.54)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: left, reward: 0.531782675862
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': 0.531782675862488, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 0.53)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 1.14609036768
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.1460903676848926, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.15)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 323
\-------------------------
Environment.reset(): Trial set up with start = (8, 7), destination = (3, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0396; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: 1.1590665158
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.159066515803021, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.16)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 1.93599729314
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 1.9359972931408647, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.94)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 2.7123750318
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.7123750318005673, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.71)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 2.91196034479
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.9119603447933597, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.91)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 2.45371202686
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.4537120268645465, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.45)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: 2.48642718367
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.486427183666227, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.49)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.0872129583
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.0872129583035703, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.09)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.75258281383
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.7525828138329422, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.75)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 2.11646680906
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.116466809061534, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.12)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.2223549307
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.2223549307009365, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.22)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 2.46996071955
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.4699607195494604, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.47)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: left, reward: 2.07686598732
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 2.0768659873169875, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 2.08)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.37097295273
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.3709729527258352, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.37)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 0.191331560711
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'left'), 'deadline': 12, 't': 13, 'action': None, 'reward': 0.19133156071059565, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.19)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 1.30140806491
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'right'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 1.301408064914682, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'right')
Agent drove right instead of forward. (rewarded 1.30)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 1.48140339023
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.481403390229105, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.48)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 0.772498317093
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 0.7724983170927848, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.77)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: -0.138801416404
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'right'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': -0.1388014164036504, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'right')
Agent drove right instead of left. (rewarded -0.14)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 2.37361563871
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 2.3736156387112963, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.37)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 2.42638970159
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 19, 'action': None, 'reward': 2.4263897015948297, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.43)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 2.29034401899
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 2.2903440189853104, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.29)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 1.8127057883
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 1.812705788301118, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.81)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 1.41093320899
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 1.410933208994188, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.41)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: 0.848667136886
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'forward'), 'deadline': 2, 't': 23, 'action': 'forward', 'reward': 0.8486671368857288, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 0.85)
4% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 324
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (5, 4), deadline = 35
Simulating trial. . .
epsilon = 0.0392; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0392; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: forward, reward: 0.0775735038705
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 35, 't': 0, 'action': 'forward', 'reward': 0.07757350387054363, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.08)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 1.80071200426
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 34, 't': 1, 'action': 'right', 'reward': 1.8007120042557163, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.80)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 1.20274234601
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 33, 't': 2, 'action': 'forward', 'reward': 1.202742346013511, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.20)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: 2.14286280463
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 32, 't': 3, 'action': 'forward', 'reward': 2.1428628046279163, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.14)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: 2.35062712159
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 31, 't': 4, 'action': 'forward', 'reward': 2.3506271215850187, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.35)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 2.90541689212
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 30, 't': 5, 'action': None, 'reward': 2.9054168921196286, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.91)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 2.09565525493
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 29, 't': 6, 'action': None, 'reward': 2.095655254925348, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.10)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: left, reward: 0.995385569544
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 28, 't': 7, 'action': 'left', 'reward': 0.995385569543705, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.00)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 1.42002807353
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 27, 't': 8, 'action': None, 'reward': 1.4200280735344508, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.42)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 0.865400011068
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'right'), 'deadline': 26, 't': 9, 'action': 'right', 'reward': 0.8654000110682807, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'right')
Agent drove right instead of forward. (rewarded 0.87)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 2.41875039198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 25, 't': 10, 'action': None, 'reward': 2.4187503919767037, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.42)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 0.966705718112
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 24, 't': 11, 'action': None, 'reward': 0.9667057181122878, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.97)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.58850250768
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 23, 't': 12, 'action': None, 'reward': 1.588502507678135, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.59)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 0.340349848463
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 22, 't': 13, 'action': 'forward', 'reward': 0.34034984846345184, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove forward instead of left. (rewarded 0.34)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: 1.20198584965
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'forward'), 'deadline': 21, 't': 14, 'action': 'forward', 'reward': 1.2019858496530142, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.20)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.23108330802
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 20, 't': 15, 'action': None, 'reward': 1.2310833080249666, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.23)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.76740595865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 16, 'action': None, 'reward': 1.7674059586454818, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.77)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: left, reward: 0.902509736941
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 18, 't': 17, 'action': 'left', 'reward': 0.9025097369405375, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 0.90)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: left, reward: 1.09863571173
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 18, 'action': 'left', 'reward': 1.098635711732983, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.10)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 0.328716610816
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 16, 't': 19, 'action': 'right', 'reward': 0.3287166108164422, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 0.33)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 0.84289191164
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 15, 't': 20, 'action': 'right', 'reward': 0.8428919116403715, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 0.84)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: right, reward: 1.80160517907
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 14, 't': 21, 'action': 'right', 'reward': 1.8016051790677097, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.80)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 2.35215949448
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 13, 't': 22, 'action': None, 'reward': 2.3521594944816413, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.35)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 0.907189855559
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 23, 'action': None, 'reward': 0.9071898555593199, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.91)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 2.33335865574
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 11, 't': 24, 'action': None, 'reward': 2.333358655739149, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.33)
29% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 2.38874495802
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 25, 'action': None, 'reward': 2.388744958022757, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.39)
26% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 2.40215862383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 26, 'action': 'right', 'reward': 2.4021586238312667, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.40)
23% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 0.775427092249
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 8, 't': 27, 'action': None, 'reward': 0.7754270922492363, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.78)
20% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 1.43056112166
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 7, 't': 28, 'action': None, 'reward': 1.4305611216610814, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.43)
17% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: 1.78902991242
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 6, 't': 29, 'action': 'forward', 'reward': 1.7890299124166258, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.79)
14% of time remaining to reach destination.
/-------------------
| Step 30 Results
\-------------------
Environment.step(): t = 30
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: forward, reward: 0.809016506679
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 5, 't': 30, 'action': 'forward', 'reward': 0.8090165066792854, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 0.81)
11% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 325
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (5, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0388; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 1.40126606042
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.4012660604171505, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 1.40)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: None, reward: 2.68198637167
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.681986371666742, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.68)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: left, reward: -40.8189300808
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 18, 't': 2, 'action': 'left', 'reward': -40.818930080811995, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.82)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: 1.61962988414
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.6196298841384529, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove forward instead of left. (rewarded 1.62)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.64070027381
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.640700273814133, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.64)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: left, reward: 1.03377216095
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.0337721609538821, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.03)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 1.08381541264
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.0838154126407817, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.08)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: right, reward: 2.49084613524
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 2.4908461352428324, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 2.49)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 326
\-------------------------
Environment.reset(): Trial set up with start = (7, 2), destination = (3, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0384; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 1.23884227631
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.2388422763081244, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.24)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.43462110541
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.4346211054073033, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.43)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.43027371535
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.4302737153529486, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 2.43)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: left, reward: 1.54437992106
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 1.5443799210586362, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.54)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 2.28228219223
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.2822821922310945, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 2.28)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: forward, reward: 1.76341600944
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.7634160094402906, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.76)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 2.75574300308
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.755743003077317, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.76)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 1.79131981969
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.7913198196949986, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.79)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 1.19167506355
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.1916750635524256, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.19)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 2.81137105487
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.8113710548662025, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.81)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 1.77681838679
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.7768183867931198, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.78)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: left, reward: 0.816100571561
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 0.8161005715607537, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.82)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.82654196205
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.8265419620483072, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.83)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: forward, reward: 2.15691059085
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 2.1569105908490256, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent followed the waypoint forward. (rewarded 2.16)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: right, reward: 0.986039716152
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 0.9860397161521454, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 0.99)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: None, reward: 1.31453138924
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.3145313892423187, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 1.31)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: None, reward: 1.79401858976
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.7940185897560428, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.79)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: left, reward: 1.59655621193
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 1.596556211930571, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 1.60)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: left, reward: 0.337251851936
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 7, 't': 18, 'action': 'left', 'reward': 0.3372518519361667, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.34)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 2.26918209778
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 2.269182097775687, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 2.27)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: -20.0175786898
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 5, 't': 20, 'action': 'right', 'reward': -20.017578689766857, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.02)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: left, reward: 1.58778980441
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 1.5877898044113274, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.59)
12% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 327
\-------------------------
Environment.reset(): Trial set up with start = (4, 7), destination = (1, 4), deadline = 30
Simulating trial. . .
epsilon = 0.0380; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 2.70656143304
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.706561433039294, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.71)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: forward, reward: -9.77216326101
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'right', 'right'), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': -9.77216326100974, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', 'right')
Agent attempted driving forward through a red light. (rewarded -9.77)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 1.37494661585
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.3749466158546102, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.37)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 2.63371598337
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.633715983366318, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.63)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 1.65239321721
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.6523932172131524, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.65)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: left, reward: 1.39405269423
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 1.3940526942260094, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.39)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 1.23936323034
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 1.2393632303391742, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.24)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 2.57726046526
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 2.5772604652557964, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.58)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: left, reward: 1.32506434118
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': 1.3250643411812322, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.33)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: forward, reward: 1.60535425421
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.6053542542113126, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.61)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 2.41832205735
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.418322057353561, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.42)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 1.93765894765
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.937658947653315, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.94)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 2.38249299769
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.3824929976907185, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.38)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: forward, reward: 0.847908906648
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 0.8479089066481154, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 0.85)
53% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 328
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (5, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0376; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0376; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.94987267773
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.949872677726016, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.95)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 2.50580388703
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.5058038870256754, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.51)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 2.43480465453
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.4348046545323134, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.43)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.71812600115
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.7181260011490926, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.72)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 2.14733084902
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.1473308490165355, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.15)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.00773649204
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.0077364920441891, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.01)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: left, reward: 0.632788378904
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 0.6327883789039126, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.63)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: 1.22099878718
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.2209987871806254, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 1.22)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 2.39089035355
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.390890353549631, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.39)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 2.74537487696
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 2.7453748769646005, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.75)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 1.51179109364
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.5117910936444419, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.51)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 2.23641939252
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 2.2364193925200393, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.24)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: 2.06559656714
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 2.0655965671397176, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.07)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: right, reward: 1.75328347521
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.75328347520923, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.75)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 329
\-------------------------
Environment.reset(): Trial set up with start = (8, 7), destination = (3, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0373; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0373; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0373; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0373; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 1.05220262465
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.0522026246506997, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.05)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 2.73702630482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.737026304816057, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 2.74)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.14638196242
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.1463819624218266, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.15)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.29612807528
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.296128075280604, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.30)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.4258424928
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.425842492797012, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove right instead of forward. (rewarded 1.43)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: left, reward: 1.80843680236
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.8084368023602382, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.81)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: 2.71454423636
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 2.7145442363564696, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.71)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: left, reward: 1.88603247692
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.8860324769175523, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.89)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 330
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (7, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0369; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0369; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0369; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0369; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: right, reward: 1.2801065038
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.2801065038024797, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.28)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: right, reward: 2.13577477746
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'right', None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.135774777464131, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'right', None)
Agent followed the waypoint right. (rewarded 2.14)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: forward, reward: 2.78274638074
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 2.782746380744091, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.78)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.21672753221
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.2167275322103062, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.22)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.5235157406
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.5235157405967226, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.52)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: left, reward: 1.69746412884
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.6974641288418773, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.70)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 2.04531586507
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.045315865066007, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.05)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 1.62729387766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.6272938776596566, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.63)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 2.40429975655
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.4042997565543702, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.40)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 2.21514987143
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.2151498714323576, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', 'left')
Agent properly idled at a red light. (rewarded 2.22)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 2.47931553138
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.4793155313761135, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.48)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: forward, reward: 1.17546799884
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.1754679988444048, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.18)
52% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 331
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (2, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0365; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 0.888576150995
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 0.8885761509947481, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 0.89)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: left, reward: 2.79360019037
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 2.7936001903697445, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.79)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 2.5331908347
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.5331908346977148, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.53)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 2.60397172492
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.6039717249245493, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.60)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 2.24488518927
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.2448851892716366, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.24)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 2.59691624508
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.596916245082351, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.60)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: left, reward: 1.68018974169
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.6801897416905673, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.68)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.8710015082
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.87100150820226, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.87)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.42277857908
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.422778579081303, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.42)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: 2.47263248254
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 2.4726324825363046, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.47)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 1.88799923457
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.8879992345670462, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.89)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: 1.02475505475
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.0247550547482338, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.02)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 332
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (7, 7), deadline = 30
Simulating trial. . .
epsilon = 0.0362; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 2.07872460258
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.0787246025752655, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.08)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 2.78365402328
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 2.7836540232771982, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.78)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 2.32451257219
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 2.3245125721921154, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.32)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 2.16364161222
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 2.163641612223252, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.16)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 2.19579939834
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 2.195799398338683, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 2.20)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: forward, reward: 1.17416311177
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.174163111771722, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.17)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 1.37530986346
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.3753098634584198, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.38)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: 0.550372229177
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 0.5503722291767458, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.55)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 0.96693193236
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 0.9669319323601937, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.97)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: None, reward: 1.44236909367
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.4423690936742521, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.44)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 2.75822001485
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 2.758220014852411, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.76)
63% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 333
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (4, 3), deadline = 35
Simulating trial. . .
epsilon = 0.0358; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0358; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0358; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0358; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 1.79764224542
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'right'), 'deadline': 35, 't': 0, 'action': 'right', 'reward': 1.7976422454163343, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'right')
Agent drove right instead of left. (rewarded 1.80)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 1.77352718814
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 34, 't': 1, 'action': None, 'reward': 1.773527188137134, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.77)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 1.52038809967
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 33, 't': 2, 'action': 'right', 'reward': 1.5203880996669854, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.52)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: 2.87601630014
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 32, 't': 3, 'action': 'forward', 'reward': 2.876016300139568, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.88)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: forward, reward: 2.27268367851
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 31, 't': 4, 'action': 'forward', 'reward': 2.272683678514533, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.27)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.99788779863
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'right'), 'deadline': 30, 't': 5, 'action': None, 'reward': 1.9978877986271386, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 2.00)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 2.00892526564
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 29, 't': 6, 'action': 'forward', 'reward': 2.0089252656380125, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.01)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 1.85436050844
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'right'), 'deadline': 28, 't': 7, 'action': 'right', 'reward': 1.8543605084408212, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'right')
Agent drove right instead of left. (rewarded 1.85)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: None, reward: 1.0812999545
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 27, 't': 8, 'action': None, 'reward': 1.0812999545007582, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.08)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: 2.01092492878
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 26, 't': 9, 'action': 'forward', 'reward': 2.010924928782895, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.01)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: forward, reward: 1.02235075864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 10, 'action': 'forward', 'reward': 1.022350758639602, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.02)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.06959485167
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 11, 'action': None, 'reward': 1.0695948516658582, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.07)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.68832875814
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 23, 't': 12, 'action': None, 'reward': 1.6883287581352018, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.69)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: forward, reward: 2.69992650314
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 22, 't': 13, 'action': 'forward', 'reward': 2.6999265031363455, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.70)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 334
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (4, 7), deadline = 30
Simulating trial. . .
epsilon = 0.0354; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0354; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 1.3916227273
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.3916227273004056, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.39)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: forward, reward: -40.0867756021
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'forward'), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': -40.08677560213561, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.09)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 2.79911905067
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 2.7991190506677146, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.80)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 1.1651635285
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 1.1651635284982507, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.17)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.14739538893
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.1473953889305026, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.15)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 2.57277106082
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 2.572771060818663, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.57)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 1.84290310792
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.8429031079226743, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.84)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 0.984454742308
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'forward'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 0.984454742307879, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.98)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: left, reward: 1.75119811841
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 22, 't': 8, 'action': 'left', 'reward': 1.7511981184112146, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.75)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: left, reward: 1.31903546002
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 1.3190354600193983, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.32)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 1.87465484716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.8746548471618283, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.87)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 2.51019251461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.510192514611262, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.51)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 1.86511480868
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.8651148086783855, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.87)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: forward, reward: 2.62567220133
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 2.62567220132587, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.63)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: forward, reward: 1.53817995874
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': 1.5381799587396265, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.54)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 335
\-------------------------
Environment.reset(): Trial set up with start = (7, 2), destination = (5, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0351; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: 2.59070837326
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.5907083732573577, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.59)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: left, reward: 0.263744177669
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 0.26374417766912706, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.26)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: right, reward: 1.2006851383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.2006851383015542, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.20)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 1.26031345166
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.2603134516615828, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.26)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: -10.6069690399
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', 'right', 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -10.606969039851725, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'right', 'left')
Agent attempted driving left through a red light. (rewarded -10.61)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: left, reward: 2.67869554804
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.678695548043059, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 2.68)
70% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 336
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (3, 6), deadline = 30
Simulating trial. . .
epsilon = 0.0347; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: 0.891745588146
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', 'forward'), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': 0.8917455881461931, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', 'forward')
Agent drove forward instead of left. (rewarded 0.89)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: right, reward: 1.01067028858
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.0106702885794334, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.01)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: left, reward: 1.98198150416
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 28, 't': 2, 'action': 'left', 'reward': 1.981981504162899, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.98)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 0.890788517533
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 0.8907885175326945, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.89)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: 2.59958626611
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 2.599586266108473, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.60)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 0.777988557301
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'right'), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 0.7779885573014448, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'right')
Agent drove right instead of forward. (rewarded 0.78)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 2.12514940578
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.125149405781758, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.13)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: left, reward: 2.31978992594
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 2.3197899259420502, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.32)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 2.04595734554
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.045957345542319, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.05)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 0.947355317423
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 0.9473553174231293, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 0.95)
67% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 337
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (5, 7), deadline = 30
Simulating trial. . .
epsilon = 0.0344; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0344; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0344; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0344; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 1.66231182503
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.662311825033883, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.66)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 1.86193029523
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.8619302952322336, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 1.86)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.96818626019
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.9681862601856925, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.97)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 1.93236406791
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 1.932364067914361, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.93)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: 0.492710681811
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 0.49271068181116806, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.49)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: 1.51725177856
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.5172517785582667, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.52)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: 1.05161011402
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 1.0516101140179723, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.05)
77% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 338
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (7, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0340; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: left, reward: 1.90728357992
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 1.9072835799222043, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.91)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 1.30149429617
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.3014942961731992, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 1.30)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 2.83266441959
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.832664419591735, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.83)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: 2.0736374063
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 2.073637406298121, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.07)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.16019753639
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.1601975363932624, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.16)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 2.56198311065
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.5619831106508553, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.56)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: left, reward: 1.62337233813
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.6233723381285783, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 1.62)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: 1.19311444664
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.1931144466351653, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.19)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: left, reward: 1.15692815848
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', 'left'), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 1.1569281584766928, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', 'left')
Agent drove left instead of forward. (rewarded 1.16)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 2.35489534315
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.3548953431535784, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.35)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 1.32031396492
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.3203139649228208, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.32)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.20620807614
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.2062080761383476, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.21)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 2.24401245044
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.2440124504375456, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.24)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 2.37934731322
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 2.379347313221771, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.38)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: -0.220667798146
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 11, 't': 14, 'action': None, 'reward': -0.22066779814591342, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.22)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 0.19004327053
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 0.1900432705296735, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent drove forward instead of right. (rewarded 0.19)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 1.54296950233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.5429695023300802, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.54)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 1.08625146902
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.086251469017305, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.09)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 2.20970746941
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 7, 't': 18, 'action': None, 'reward': 2.2097074694086354, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.21)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 1.24217574934
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 1.2421757493411105, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.24)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: left, reward: 2.23512626708
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 5, 't': 20, 'action': 'left', 'reward': 2.2351262670789556, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.24)
16% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 339
\-------------------------
Environment.reset(): Trial set up with start = (7, 4), destination = (5, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0337; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 1.8844934712
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.8844934712029515, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.88)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: forward, reward: 2.89211715198
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 2.8921171519790922, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.89)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: 1.61945528515
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.6194552851454627, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.62)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 2.81132786098
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.811327860977502, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.81)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 1.29896143696
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.2989614369635452, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.30)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: forward, reward: 2.08886891455
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 2.0888689145473833, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.09)
70% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 340
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (4, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0334; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: left, reward: 0.0993090391784
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 0.09930903917836131, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', None)
Agent drove left instead of right. (rewarded 0.10)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: None, reward: 0.546548586479
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 0.5465485864791173, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.55)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: right, reward: 1.67723678824
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.6772367882413277, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 1.68)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: left, reward: 0.968192899468
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 0.9681928994681226, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 0.97)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: 1.07090394028
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 1.0709039402846825, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.07)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 1.13414594226
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.134145942257503, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.13)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 1.17439133824
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.1743913382403457, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 1.17)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: forward, reward: 1.60658250858
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.6065825085826153, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.61)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 1.47614608992
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.4761460899194365, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.48)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 0.0519061740439
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.05190617404394171, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.05)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.45493311599
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.4549331159884642, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.45)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: left, reward: 2.26368239236
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 2.263682392364825, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.26)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 1.14453392069
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.1445339206886547, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.14)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: left, reward: 1.55998711114
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 1.559987111135735, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.56)
44% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 341
\-------------------------
Environment.reset(): Trial set up with start = (4, 3), destination = (7, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0330; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0330; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0330; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 1.61287239925
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.612872399251009, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 1.61)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: 2.46447080826
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.464470808258037, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.46)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 1.51620079911
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.5162007991109727, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.52)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 2.7219153107
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.721915310702224, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.72)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 1.92950574103
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.9295057410348688, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.93)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 1.77870157774
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'right'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.7787015777362778, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'right')
Agent drove right instead of forward. (rewarded 1.78)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 0.469629586468
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 0.4696295864682578, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.47)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.66693216971
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.6669321697135182, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.67)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.31009265994
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.3100926599411307, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.31)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.32983657628
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.329836576279093, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.33)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: forward, reward: -0.218538822304
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': -0.21853882230373511, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded -0.22)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 1.72709466264
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.7270946626378951, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.73)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 1.5311855115
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.5311855115016022, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.53)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 0.873623250499
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.8736232504989684, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 0.87)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.79277838475
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'right'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.7927783847474006, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'right')
Agent properly idled at a red light. (rewarded 1.79)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 2.1945756648
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.194575664797069, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.19)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: 1.12072520221
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.1207252022059178, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.12)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 0.636054282796
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.6360542827959128, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.64)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 0.915077218833
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.9150772188326326, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.92)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 1.36668478637
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.366684786372103, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.37)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 342
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (3, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0327; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0327; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 2.00094580902
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.0009458090167627, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.00)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 1.3728399029
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.3728399028974239, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.37)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 2.27325815254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.27325815254439, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.27)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 1.44355061662
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.4435506166165004, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.44)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 1.66146652935
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'forward'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.6614665293476762, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'forward')
Agent drove right instead of left. (rewarded 1.66)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.06416288878
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.0641628887790984, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.06)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.31713692155
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.3171369215470712, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.32)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: left, reward: 1.50144513358
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.5014451335848402, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.50)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 1.22339408095
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.223394080952206, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.22)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: forward, reward: 0.375004340502
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.37500434050197096, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.38)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: None, reward: 1.14451316635
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.1445131663545036, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.14)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: None, reward: 2.2082291367
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.208229136702558, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.21)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: None, reward: 1.54636625882
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.5463662588201323, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.55)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: left, reward: 1.98408936327
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.9840893632672385, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 1.98)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 2.03539386305
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.0353938630535042, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.04)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.32677906498
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.3267790649770068, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.33)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: 2.15559261471
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 2.1555926147062197, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.16)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 0.78197727308
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.7819772730797713, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.78)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 1.94099283722
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.940992837222969, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.94)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 1.1369532231
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.1369532230998292, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.14)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 343
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (5, 2), deadline = 30
Simulating trial. . .
epsilon = 0.0324; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0324; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0324; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 0.898879955656
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': 0.8988799556558658, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent drove forward instead of right. (rewarded 0.90)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 2.18140010607
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 2.1814001060683603, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.18)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 2.46263181997
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 2.462631819965398, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 2.46)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 2.02733215077
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.0273321507708055, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.03)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.58537223865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.5853722386461329, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.59)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 2.93244973469
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 2.932449734687421, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent followed the waypoint forward. (rewarded 2.93)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.76202265911
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.7620226591144872, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.76)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.3803536433
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.3803536433012573, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.38)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.18318293322
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.1831829332230672, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.18)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 1.07878474543
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.078784745429314, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.08)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: forward, reward: 0.934356270349
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 0.9343562703489752, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 0.93)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: forward, reward: 0.13989708917
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'forward'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 0.13989708916994104, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.14)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: 1.68126897422
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 18, 't': 12, 'action': 'left', 'reward': 1.6812689742151425, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.68)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: forward, reward: 1.03839513657
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.0383951365654296, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.04)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 2.00591438161
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 14, 'action': None, 'reward': 2.0059143816137546, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.01)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.86440950482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.8644095048235907, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.86)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: left, reward: 2.05453342952
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 2.054533429522034, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.05)
43% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 344
\-------------------------
Environment.reset(): Trial set up with start = (7, 4), destination = (1, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0321; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0321; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 2.10942823605
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.109428236050591, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.11)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 0.867073438807
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.8670734388074745, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.87)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: 1.53698564091
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.5369856409090976, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.54)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.35484644202
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.35484644202151, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.35)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 1.17481242569
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.1748124256883123, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'left')
Agent drove right instead of forward. (rewarded 1.17)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 1.2097539279
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.2097539278969232, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.21)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 2.24517009227
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.2451700922681965, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.25)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: 1.73216523652
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.732165236519982, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.73)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.01797687687
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.0179768768749256, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.02)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 0.906168279352
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.9061682793521013, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.91)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 0.826716389502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 0.8267163895023029, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 0.83)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: 0.752026815946
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 0.7520268159460668, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.75)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 1.95818192667
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.9581819266723195, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.96)
35% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 345
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (6, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0317; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0317; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: forward, reward: 0.608343327095
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 0.6083433270949199, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove forward instead of left. (rewarded 0.61)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.26584179414
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.265841794138143, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.27)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.62152402484
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.6215240248395895, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.62)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 0.99705252252
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 0.9970525225201659, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.00)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: left, reward: 2.50946678875
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 2.509466788748557, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.51)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 1.45542735131
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.4554273513098435, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.46)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 2.65467335853
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.6546733585321434, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.65)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 2.18253515302
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.1825351530245203, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.18)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 1.84027153067
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.8402715306689865, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 1.84)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: left, reward: 2.36154969443
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 2.3615496944303187, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.36)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 1.11775263661
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.117752636610688, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.12)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 2.51852196737
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.5185219673732324, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', None)
Agent properly idled at a red light. (rewarded 2.52)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 1.47855661028
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', 'left'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.478556610280143, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', 'left')
Agent drove right instead of forward. (rewarded 1.48)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: 1.29365263625
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 1.293652636248421, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.29)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: 1.97319969982
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.9731996998215267, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.97)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: 2.45669285372
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'right'), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.456692853723294, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 2.46)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: forward, reward: 1.19666387736
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 1.1966638773551495, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.20)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: 2.41680567131
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 2.4168056713131465, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.42)
28% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 346
\-------------------------
Environment.reset(): Trial set up with start = (3, 2), destination = (2, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0314; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0314; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0314; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0314; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 2.43920039883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.4392003988250752, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.44)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.05612553626
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.0561255362587514, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.06)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 2.06037406458
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.06037406458279, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.06)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: 2.60959502855
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 2.6095950285548755, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.61)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 2.09116186897
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.091161868969742, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.09)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 1.61268610252
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.6126861025243486, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.61)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: forward, reward: -0.103812157007
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -0.10381215700709134, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded -0.10)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.22581096618
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.22581096618069, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.23)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 0.894998186459
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 0.8949981864590115, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.89)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 1.03239390826
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.0323939082557, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'forward')
Agent drove right instead of left. (rewarded 1.03)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 1.97001661247
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.9700166124664094, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.97)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 0.649378433878
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.6493784338782596, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove right instead of left. (rewarded 0.65)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 0.729234029139
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.7292340291391919, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 0.73)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 0.76204928577
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.7620492857701464, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 0.76)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 2.24224792474
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.242247924739707, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.24)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 1.05308590706
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.0530859070598861, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.05)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 1.47740031901
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.4774003190137723, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.48)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: -0.612322185885
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': -0.6123221858846549, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded -0.61)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: 0.945577436959
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': 0.9455774369590395, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 0.95)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 1.80576773598
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.8057677359766633, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.81)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 347
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (4, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0311; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0311; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 1.01833369323
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.0183336932340028, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.02)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.03688955215
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.0368895521511736, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.04)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.50996573387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.509965733873677, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.51)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.2129056751
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.2129056751026974, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.21)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 1.18353308401
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 1.1835330840106801, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.18)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 2.32406315649
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.3240631564909755, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.32)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 1.09946570927
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.0994657092726494, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.10)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 2.00525213836
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.0052521383557456, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.01)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 2.60892542814
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.608925428138896, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.61)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 0.905860308778
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 0.9058603087775492, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.91)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 2.54424372198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.544243721978397, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.54)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: left, reward: 1.14196542159
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.1419654215853596, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.14)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: 0.968667744122
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 0.968667744122188, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 0.97)
48% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 348
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (5, 4), deadline = 25
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 2.48838561583
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.488385615826717, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.49)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 0.000137273319875
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.00013727331987478664, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.00)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: left, reward: 2.47146578486
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 2.4714657848646233, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.47)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.85663054939
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.856630549392297, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.86)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 1.30884771255
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.3088477125484357, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.31)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: 1.49019892748
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.4901989274776046, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.49)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: 2.12484005421
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.124840054213986, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 2.12)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 2.11414210286
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.11414210285953, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.11)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 2.12023027716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.120230277159556, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.12)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 1.38381722348
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.3838172234757256, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.38)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: left, reward: 1.96653074414
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 1.9665307441429067, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.97)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 2.56061899827
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.5606189982658494, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.56)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 0.815222686695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 13, 't': 12, 'action': None, 'reward': 0.8152226866946519, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 0.82)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 1.32540753451
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'forward'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.3254075345056129, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.33)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.03957024894
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.0395702489355074, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.04)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 2.42871337797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.4287133779697267, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.43)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.74721403395
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.7472140339538589, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.75)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: left, reward: 1.8867360084
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 1.8867360084042872, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.89)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 1.84345850354
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.843458503544645, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.84)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 1.67129205469
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 19, 'action': None, 'reward': 1.6712920546919132, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.67)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 2.31828611782
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 5, 't': 20, 'action': None, 'reward': 2.3182861178241634, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.32)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: forward, reward: -0.275401096983
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': -0.27540109698346626, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove forward instead of left. (rewarded -0.28)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: left, reward: 0.982346301152
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 3, 't': 22, 'action': 'left', 'reward': 0.9823463011521962, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.98)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: 1.10618157945
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'right'), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.1061815794532688, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.11)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: 1.21018380557
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.2101838055660816, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.21)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 349
\-------------------------
Environment.reset(): Trial set up with start = (7, 3), destination = (2, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0305; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 0.49106262604
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 0.4910626260404416, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 0.49)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 1.18791125087
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.187911250868877, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.19)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.71863335287
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.7186333528673423, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.72)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.32126708128
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.32126708128441, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.32)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 2.80671269976
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.8067126997603067, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.81)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 2.00004590289
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.000045902891661, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.00)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.38706950597
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.3870695059699159, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.39)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.70053924084
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.700539240839208, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.70)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.19782762219
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.1978276221864887, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.20)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 1.85171007957
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.8517100795671921, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.85)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: left, reward: 1.05073014047
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 1.0507301404707725, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.05)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: -10.0973573053
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', 'right', 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': -10.097357305264735, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', 'left')
Agent attempted driving forward through a red light. (rewarded -10.10)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: right, reward: 2.64229447726
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 2.6422944772604087, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.64)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: right, reward: 2.23298899468
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 2.2329889946816337, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.23)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 2.60026366797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.600263667971688, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.60)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 2.56659314418
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.5665931441839342, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.57)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 1.03362517962
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.0336251796175826, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.03)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: left, reward: 1.08124886216
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 1.081248862162465, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.08)
28% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 350
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (5, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0302; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.37572456332
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.375724563317724, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.38)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.15796494359
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.1579649435924115, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.16)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.70960507413
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.7096050741269153, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.71)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.15669981758
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.156699817583769, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.16)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.28382551679
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.2838255167916346, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.28)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: left, reward: 0.93014002619
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 0.9301400261898203, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.93)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: 2.60016598333
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.6001659833329565, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.60)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.70615201783
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.706152017828822, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.71)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: forward, reward: 1.78114513304
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.7811451330393064, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.78)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: forward, reward: 1.45809642741
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.458096427410789, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.46)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 351
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (4, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0299; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: -39.5691757904
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', 'right'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -39.56917579040205, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.57)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 1.31625750893
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.3162575089317854, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.32)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: forward, reward: 1.93276453729
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.9327645372935356, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.93)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: 0.955639819033
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 0.9556398190329267, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.96)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: 1.621061374
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.6210613739986715, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.62)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: 2.68339335671
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 2.683393356714922, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent followed the waypoint forward. (rewarded 2.68)
70% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 352
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (2, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0296; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: 1.14413031052
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.1441303105222969, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove left instead of right. (rewarded 1.14)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 2.89373159768
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.893731597683457, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.89)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 2.51375897288
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.5137589728809586, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.51)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 1.11943883083
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.1194388308301804, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.12)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 1.13060966762
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.1306096676161408, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.13)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: left, reward: 0.625731958664
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 0.6257319586636645, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.63)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: right, reward: 2.6182393253
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.618239325301743, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 2.62)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: 0.924353183226
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 0.9243531832262786, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.92)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 1.87549954604
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.8754995460365378, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.88)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 1.93598001303
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.935980013025525, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 1.94)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 353
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (3, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0293; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0293; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0293; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0293; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: 2.30651937462
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.30651937462264, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.31)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: 0.504903316408
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.5049033164075071, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'left')
Agent drove right instead of forward. (rewarded 0.50)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: left, reward: 2.70586851962
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 2.7058685196193197, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 2.71)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 1.76674429004
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.7667442900417485, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.77)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 0.936070884754
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.9360708847537053, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'left')
Agent drove right instead of forward. (rewarded 0.94)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: left, reward: 1.53134200244
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.5313420024425166, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.53)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.42663575614
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.4266357561386538, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.43)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 2.7354293935
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.7354293935040555, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.74)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.42527073733
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.4252707373286366, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.43)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.41996493806
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.4199649380578543, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.42)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.82071764107
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.8207176410730024, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.82)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: left, reward: 2.16053925717
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 2.16053925716548, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.16)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 354
\-------------------------
Environment.reset(): Trial set up with start = (7, 6), destination = (4, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0290; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0290; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0290; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 2.82967728117
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.8296772811654933, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 2.83)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: forward, reward: 2.3768779975
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 2.376877997496513, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 2.38)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.44199502576
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.441995025763111, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.44)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 1.01774529265
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.017745292646423, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.02)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.37257016211
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.3725701621146933, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.37)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: forward, reward: 1.24591665304
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.2459166530434118, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.25)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: right, reward: 2.07141640509
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.0714164050922124, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.07)
65% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 355
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (8, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0287; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.81010868294
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.8101086829396766, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.81)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 2.36167707469
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.3616770746876425, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.36)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 2.78512326816
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.7851232681611253, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.79)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 2.42293393525
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.4229339352546297, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.42)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 2.16217739902
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.1621773990215347, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.16)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 2.57009101596
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.5700910159618804, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.57)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: left, reward: 2.83197065356
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.831970653555195, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.83)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 1.17023397535
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.170233975348731, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.17)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 1.30922295311
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.309222953106993, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.31)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: 1.90834919242
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.9083491924237752, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.91)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 2.15959680515
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.1595968051547345, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.16)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 2.13064634958
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 2.1306463495781767, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.13)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 2.58709317196
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 2.587093171958871, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.59)
35% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 356
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (8, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0284; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 1.91829031453
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'right'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.9182903145343764, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.92)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 2.96180852709
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.9618085270924044, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.96)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 2.01471433513
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.0147143351263423, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.01)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 2.84689522391
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.846895223913447, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.85)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 1.21414852142
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.2141485214229815, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.21)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: right, reward: 0.00705821645071
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'right'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.007058216450707033, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'right')
Agent drove right instead of forward. (rewarded 0.01)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: 1.17249193974
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.172491939744649, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 1.17)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 2.53201737238
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 2.5320173723752513, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.53)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 1.5973971597
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.5973971597008199, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.60)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 1.80699177537
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.8069917753693054, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.81)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 2.67223375117
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.6722337511670764, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.67)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: forward, reward: 2.34797410439
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 2.347974104393402, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 2.35)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.43502313588
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.4350231358834034, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 1.44)
48% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 357
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (4, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0282; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 2.71601838148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.71601838148215, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 2.72)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 1.70217692582
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.702176925815265, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.70)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 2.1416679914
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.14166799139707, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.14)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 2.66248888008
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 2.6624888800849504, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.66)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 2.64531540131
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.6453154013112723, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.65)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: 1.87506057756
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.8750605775557765, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 1.88)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 2.27931623504
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.2793162350439613, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.28)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 1.05158880976
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.0515888097609716, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.05)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 0.183798793327
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.18379879332661464, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.18)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: forward, reward: 0.897688103114
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 0.8976881031137693, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.90)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 1.00146722947
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.0014672294659737, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 1.00)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 2.36533829473
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.365338294725385, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.37)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 1.21965604821
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.2196560482136092, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.22)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: 1.92116570195
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 1.9211657019473152, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.92)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 0.298734172198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 0.2987341721982161, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 0.30)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: 1.64454682444
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.644546824442984, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.64)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: 1.73196386662
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 1.731963866617942, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.73)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: left, reward: 2.5176912942
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 2.517691294196995, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.52)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: -10.2837095858
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': -10.283709585835886, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent attempted driving forward through a red light. (rewarded -10.28)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: forward, reward: 2.3333067048
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 2.3333067048023466, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.33)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 358
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (7, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0279; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 1.86989156401
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.8698915640095999, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.87)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: left, reward: 1.82706827578
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 1.8270682757803214, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 1.83)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 2.33891281748
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 2.338912817477622, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.34)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 2.87520653185
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.875206531846441, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.88)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 2.44099568048
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.440995680483123, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.44)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 1.04644262389
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.046442623885419, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.05)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 2.63119552383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.6311955238263662, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.63)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 1.40526493069
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.4052649306906144, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.41)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 1.90130830137
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 1.9013083013665066, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.90)
64% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 359
\-------------------------
Environment.reset(): Trial set up with start = (4, 3), destination = (1, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0276; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 2.29663888742
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.2966388874176697, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.30)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: 2.87973224645
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 2.879732246445149, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.88)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 2.10847510249
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.108475102492675, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.11)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.09283224587
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.092832245868951, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.09)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 2.5890782905
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.589078290495765, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.59)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 1.1177985436
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.1177985435956599, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.12)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.63353044441
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.63353044441126, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.63)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 0.68851009543
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 0.6885100954304542, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.69)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 1.79307019824
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.7930701982412949, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 1.79)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 360
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (7, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0273; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0273; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: 2.61135566117
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.6113556611710207, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.61)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 0.647293922461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'right'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.6472939224612867, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'right')
Agent drove right instead of forward. (rewarded 0.65)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 1.58587028236
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.5858702823609503, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.59)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 2.50188730091
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.501887300913925, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.50)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 2.52451393427
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.524513934272775, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.52)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: 1.15679474042
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.1567947404163375, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.16)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: -9.97258338225
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', 'forward'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -9.972583382253838, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.97)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: 2.57410952985
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.5741095298520076, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.57)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 2.37740878724
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.377408787243529, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.38)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 1.18023601171
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.1802360117071797, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.18)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: 1.97272574253
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.9727257425347933, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.97)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 361
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (1, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0271; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: 2.11562684636
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.115626846356548, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.12)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 2.67032371498
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.670323714980652, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.67)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 1.31113345918
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.311133459183825, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.31)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 2.53803189125
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.5380318912489117, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 2.54)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 1.95533214
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.9553321399982189, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.96)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 2.78091988245
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.7809198824504624, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.78)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: 0.186109976252
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.1861099762518612, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 0.19)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: left, reward: 0.936330166364
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 0.9363301663642616, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 0.94)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 2.61729450379
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 2.617294503793671, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.62)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 2.43217101382
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.4321710138190573, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.43)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 1.2206981453
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.2206981453010315, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.22)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 1.16000781303
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.1600078130272398, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.16)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 2.13352834048
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.133528340479952, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.13)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 0.130090553224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.13009055322367702, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.13)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 1.33977633727
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.339776337271831, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.34)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 0.878832742758
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.8788327427579468, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 0.88)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 1.17566782637
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.1756678263670863, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.18)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: left, reward: 0.935812160568
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 0.9358121605684278, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.94)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: left, reward: 2.2133852605
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': 2.2133852605021334, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.21)
5% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 362
\-------------------------
Environment.reset(): Trial set up with start = (7, 3), destination = (3, 7), deadline = 30
Simulating trial. . .
epsilon = 0.0268; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0268; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 1.26461129724
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.264611297238012, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.26)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 1.9573579056
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.9573579056037957, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.96)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.73777728233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.7377772823339164, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.74)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 0.996358105466
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 0.9963581054662574, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.00)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 2.52304134706
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 2.523041347061043, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.52)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.19223145873
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.192231458731001, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.19)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 2.64769814008
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.647698140080591, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.65)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 1.3073842873
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 1.3073842873029664, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.31)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 2.90225587271
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.9022558727073386, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.90)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.59517644443
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.5951764444293834, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.60)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 2.57424604621
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.5742460462148236, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.57)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 0.064627037828
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 0.06462703782800328, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove right instead of left. (rewarded 0.06)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 1.41733190504
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.417331905038111, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.42)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: forward, reward: 1.33692882688
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.3369288268830177, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.34)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 2.47522866147
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 2.475228661474193, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.48)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 0.957902701618
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 0.9579027016182933, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.96)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 0.780874105669
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 14, 't': 16, 'action': None, 'reward': 0.7808741056685731, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.78)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: 0.840117196003
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 0.8401171960025942, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.84)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: left, reward: 1.68274652066
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 12, 't': 18, 'action': 'left', 'reward': 1.6827465206568402, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.68)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 1.38882321048
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 11, 't': 19, 'action': None, 'reward': 1.3888232104822904, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.39)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: left, reward: -9.20174365649
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 10, 't': 20, 'action': 'left', 'reward': -9.201743656492619, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent attempted driving left through a red light. (rewarded -9.20)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: forward, reward: 1.30257528528
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 1.3025752852831614, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.30)
27% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 363
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (8, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0265; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.01914084708
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.0191408470805485, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.02)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 1.55900378334
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.5590037833427282, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.56)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 1.85825937563
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.8582593756258992, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.86)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 2.01101303315
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.0110130331517944, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.01)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: left, reward: 2.1538611971
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 2.1538611971011954, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 2.15)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 1.57252667502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.5725266750172369, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.57)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 2.29745277117
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.2974527711730204, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.30)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: 1.75404787562
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.7540478756150009, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.75)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: 1.49799242136
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.4979924213582292, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove left instead of forward. (rewarded 1.50)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 1.01624630153
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.0162463015281347, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.02)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 1.20420355876
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.2042035587623945, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.20)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 364
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (5, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0263; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: 2.15945895103
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 2.159458951027352, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.16)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 2.51739344134
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.5173934413371755, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.52)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 2.90572886859
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.905728868593651, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.91)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: 2.59058546723
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.5905854672337387, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.59)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 2.83497719073
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.834977190729532, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.83)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: forward, reward: 1.30954561237
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.3095456123696738, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.31)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: left, reward: 2.83843768578
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 2.83843768578173, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.84)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 1.12348096596
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.1234809659612224, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.12)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 2.56776540045
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.5677654004487214, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.57)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: forward, reward: 0.898393042231
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 0.8983930422305666, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.90)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 365
\-------------------------
Environment.reset(): Trial set up with start = (7, 2), destination = (8, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0260; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0260; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 2.63882012337
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.638820123373872, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.64)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 2.00842099896
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.008420998958253, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.01)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.65795655517
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.6579565551680822, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.66)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: 2.92450241978
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 2.924502419783421, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.92)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.46551347539
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.4655134753931824, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.47)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.82208635382
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.8220863538175973, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.82)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.32743760736
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.327437607362563, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.33)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: 1.31038476156
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.310384761562955, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.31)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.2372181425
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.2372181424966557, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.24)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.3972691229
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.3972691228963114, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.40)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.23383753567
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.2338375356713644, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.23)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: forward, reward: 2.49969847944
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 2.4996984794387918, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.50)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 1.40879579369
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.408795793686401, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.41)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: forward, reward: 0.756988372885
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.756988372884593, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.76)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 366
\-------------------------
Environment.reset(): Trial set up with start = (5, 5), destination = (1, 3), deadline = 30
Simulating trial. . .
epsilon = 0.0257; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0257; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: -5.82342692769
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': -5.8234269276907575, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.82)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 2.22252391829
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 2.2225239182861865, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.22)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 2.89580882989
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.8958088298943228, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.90)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 2.93455149098
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.9345514909836696, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.93)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: 2.53189635284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 2.531896352838129, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.53)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.29678060028
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.2967806002800182, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.30)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 2.00400568523
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.0040056852258434, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.00)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 2.15183535691
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 2.1518353569065223, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.15)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: forward, reward: 2.24526506818
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': 2.2452650681820137, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', None)
Agent followed the waypoint forward. (rewarded 2.25)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 2.28392431308
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 9, 'action': None, 'reward': 2.2839243130794467, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.28)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.2774637861
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.2774637861028362, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.28)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: forward, reward: 1.34976674161
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.3497667416128605, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.35)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: right, reward: 1.5485604084
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 1.5485604084011242, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.55)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: forward, reward: -9.55592005972
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'right', 'forward'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': -9.555920059718023, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.56)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: right, reward: 1.66405713445
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.6640571344518458, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.66)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 2.30265278801
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': None, 'reward': 2.3026527880102665, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.30)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 2.64414103665
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 2.6441410366500158, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.64)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: None, reward: -4.94128767111
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', 'right', 'forward'), 'deadline': 13, 't': 17, 'action': None, 'reward': -4.941287671111815, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.94)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: right, reward: 2.00192147562
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 2.0019214756185484, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.00)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: forward, reward: 0.374747356279
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': 0.37474735627939293, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 0.37)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: 1.05330833576
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', None), 'deadline': 10, 't': 20, 'action': 'forward', 'reward': 1.0533083357564814, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', None)
Agent drove forward instead of left. (rewarded 1.05)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 1.17870055269
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 21, 'action': None, 'reward': 1.1787005526876642, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.18)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 0.998658964346
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 22, 'action': None, 'reward': 0.9986589643457386, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.00)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: left, reward: 0.626440965074
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 7, 't': 23, 'action': 'left', 'reward': 0.6264409650738043, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 0.63)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: 1.30693909652
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 6, 't': 24, 'action': 'left', 'reward': 1.3069390965196979, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 1.31)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 0.975786043441
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 5, 't': 25, 'action': None, 'reward': 0.9757860434406365, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 0.98)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 1.12709836259
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', None), 'deadline': 4, 't': 26, 'action': 'right', 'reward': 1.1270983625875044, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', None)
Agent drove right instead of forward. (rewarded 1.13)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: 1.44251893507
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 3, 't': 27, 'action': None, 'reward': 1.4425189350728982, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.44)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: 0.344824347506
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 2, 't': 28, 'action': None, 'reward': 0.34482434750583746, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 0.34)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: left, reward: 0.976923040186
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 1, 't': 29, 'action': 'left', 'reward': 0.9769230401860103, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.98)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 367
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (4, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0255; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 1.6730361422
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.6730361421952247, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.67)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 2.97583241106
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.9758324110576297, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', None)
Agent properly idled at a red light. (rewarded 2.98)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 1.97197436127
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.971974361267586, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.97)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: 1.5980174952
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 1.5980174952026822, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.60)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.23279980193
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.2327998019300037, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.23)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 0.863074241547
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 0.8630742415467895, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.86)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 1.53896996462
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.5389699646180903, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.54)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 0.979636147947
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 0.9796361479466078, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 0.98)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.82088044247
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.8208804424728573, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.82)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 2.31873206639
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.318732066387374, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.32)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 2.0690860261
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.069086026101504, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.07)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: 0.828306903515
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 0.8283069035152311, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.83)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.62611235898
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.626112358980816, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.63)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 1.38573402978
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.38573402978423, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.39)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.38236603659
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.382366036593091, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.38)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: left, reward: -9.81587436826
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', 'forward'), 'deadline': 10, 't': 15, 'action': 'left', 'reward': -9.815874368256932, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -9.82)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: -0.032519571842
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'forward'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': -0.032519571841961414, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded -0.03)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 0.782173878537
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 0.7821738785368373, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.78)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 1.34631671657
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.3463167165672105, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.35)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 2.2699294184
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 6, 't': 19, 'action': None, 'reward': 2.2699294183961163, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.27)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 0.557545860797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.5575458607968817, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.56)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: left, reward: 2.05409715687
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 2.0540971568733313, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.05)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 0.525936112418
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'left'), 'deadline': 3, 't': 22, 'action': None, 'reward': 0.5259361124181325, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.53)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: left, reward: 1.28891170002
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 2, 't': 23, 'action': 'left', 'reward': 1.2889117000205135, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.29)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 0.58792564212
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': 0.5879256421199215, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.59)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 368
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (7, 2), deadline = 30
Simulating trial. . .
epsilon = 0.0252; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: 2.28250482175
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 2.282504821746979, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 2.28)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 1.24802055174
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 1.2480205517355614, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.25)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 1.18190826496
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 1.181908264956998, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded 1.18)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: None, reward: 2.96800351042
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.9680035104166143, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.97)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: None, reward: 1.41901059715
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.4190105971505764, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.42)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: right, reward: 1.34560403973
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.3456040397321272, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 1.35)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 2.4740353563
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.4740353562999307, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.47)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.14183011828
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.1418301182751625, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.14)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 2.68739612973
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.6873961297328353, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.69)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.53614018868
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.5361401886801749, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.54)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 2.15770231099
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.1577023109902536, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.16)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: left, reward: 2.6770560757
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 2.677056075695734, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.68)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: forward, reward: -39.0734521814
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': -39.073452181385285, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.07)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: left, reward: 1.94609794495
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 1.9460979449489393, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.95)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 1.12032009397
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'right'), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.120320093973321, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.12)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 0.867490475459
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'forward'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 0.867490475459041, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.87)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 2.72097457089
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 14, 't': 16, 'action': None, 'reward': 2.7209745708931194, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.72)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 0.965811507786
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 13, 't': 17, 'action': None, 'reward': 0.9658115077860776, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 0.97)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 1.42051583226
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.4205158322649958, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.42)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 1.44810935155
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 11, 't': 19, 'action': None, 'reward': 1.4481093515522239, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.45)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 1.09369679377
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 10, 't': 20, 'action': None, 'reward': 1.0936967937658935, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.09)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 0.888882475255
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 0.88888247525507, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent drove right instead of left. (rewarded 0.89)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 0.908269413367
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': 0.9082694133672984, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent drove forward instead of right. (rewarded 0.91)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 2.37368316261
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 2.3736831626107486, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 2.37)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: forward, reward: 0.800560887369
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 6, 't': 24, 'action': 'forward', 'reward': 0.8005608873691354, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 0.80)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 1.31002516354
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'right'), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 1.3100251635431694, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 1.31)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.96076135943
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 4, 't': 26, 'action': None, 'reward': 1.9607613594295699, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.96)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.02557715325
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 3, 't': 27, 'action': None, 'reward': 1.0255771532450524, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.03)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.09483612368
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 2, 't': 28, 'action': None, 'reward': 1.0948361236759938, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.09)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.70062406841
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 1, 't': 29, 'action': None, 'reward': 1.7006240684143208, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.70)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 369
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (2, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0250; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: forward, reward: 1.89124938038
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.8912493803784107, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.89)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: 0.198926294591
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 0.19892629459076094, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.20)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.22470561974
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.2247056197427841, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.22)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 2.54591073967
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.5459107396719243, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.55)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 0.157609822397
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'forward'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.1576098223971164, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.16)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: 2.60469647733
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.6046964773284405, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.60)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: 1.81044999437
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.8104499943665284, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 1.81)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: 1.37614490479
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.376144904793307, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'left')
Agent followed the waypoint forward. (rewarded 1.38)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 2.19527679386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.195276793861612, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.20)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 1.90450854326
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.9045085432583917, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.90)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 0.794286427401
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.7942864274009718, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 0.79)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 370
\-------------------------
Environment.reset(): Trial set up with start = (7, 2), destination = (4, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0247; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0247; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: 2.4902715835
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.490271583499746, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 2.49)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.11837065812
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.1183706581165318, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.12)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.08362849523
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.0836284952309916, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.08)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.99524384632
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.9952438463243902, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.00)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.97862681618
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.9786268161834528, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.98)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.41032776532
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.4103277653237822, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.41)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: 0.907066971359
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 0.9070669713593558, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent followed the waypoint forward. (rewarded 0.91)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 1.93503798542
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.9350379854235225, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.94)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 1.69516722755
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.6951672275499035, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.70)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 0.988770652056
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.9887706520555497, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.99)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: 2.06625645216
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 2.0662564521576403, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.07)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 1.67729248849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.6772924884884104, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.68)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 1.35179922921
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.3517992292148486, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.35)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: left, reward: 2.33049000536
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 2.330490005357225, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.33)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 371
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (2, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0245; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0245; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: -5.05116467118
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'right'), 'deadline': 20, 't': 0, 'action': None, 'reward': -5.051164671184362, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.05)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: 2.8410531521
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 2.8410531521012863, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 2.84)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.938485171
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.9384851709975583, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.94)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.18243180478
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.182431804779776, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.18)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.3873087423
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.3873087423035309, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.39)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.18762250741
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.1876225074065723, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.19)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 1.83945747461
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.839457474607885, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.84)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: 2.68179503765
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.681795037650992, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.68)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.96308177126
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.9630817712584534, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.96)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 0.0204079863399
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.020407986339865403, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.02)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: left, reward: 1.83355752387
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.833557523869106, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.83)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 372
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (5, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0242; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.21200976273
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.2120097627321926, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.21)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: 2.52712528624
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 2.527125286239841, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 2.53)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: 2.17605038497
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 2.1760503849652264, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.18)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 1.30042383547
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.3004238354729956, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.30)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: 2.24283291226
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.242832912256198, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.24)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.58524385925
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.585243859245506, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.59)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: right, reward: 2.80800870225
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.8080087022474647, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.81)
72% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 373
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (7, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0240; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0240; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: left, reward: -39.3327115955
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -39.33271159554666, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.33)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 2.46211224971
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.4621122497141723, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.46)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: left, reward: -39.1055943525
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'right', 'forward', 'forward'), 'deadline': 18, 't': 2, 'action': 'left', 'reward': -39.10559435249233, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.11)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: 1.15319639378
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.153196393779687, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.15)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 2.81797507732
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.8179750773154755, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.82)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 1.65770459616
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.6577045961561312, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.66)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: right, reward: 0.593362740374
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.5933627403736037, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.59)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 2.71417800191
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.7141780019111, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.71)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 1.14741486766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.14741486766228, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.15)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: right, reward: 0.0348503355246
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'right'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.03485033552455219, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'right')
Agent drove right instead of left. (rewarded 0.03)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 2.20903658628
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 2.2090365862845154, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.21)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 2.38366768589
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', 'forward'), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.3836676858904475, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.38)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: 0.865604001115
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.8656040011150019, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.87)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: forward, reward: 1.94372555079
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.9437255507881377, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.94)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 1.01205396369
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 1.0120539636885528, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.01)
25% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 374
\-------------------------
Environment.reset(): Trial set up with start = (2, 7), destination = (5, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0238; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0238; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0238; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 2.10814544118
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.108145441179039, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.11)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 2.57699865838
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.5769986583835514, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.58)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.55082166053
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.5508216605321798, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 1.55)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.5791374579
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.579137457904904, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.58)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.91414885978
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.9141488597809357, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.91)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: right, reward: 1.6096037282
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.6096037282043576, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.61)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: right, reward: 1.849550024
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.849550024001359, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 1.85)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 2.05262246714
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.0526224671442153, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 2.05)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 2.46581814648
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.465818146476626, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.47)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.9836149291
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.9836149290964484, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.98)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: left, reward: 2.66542208003
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 2.6654220800254156, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.67)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 2.69290485165
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 2.6929048516503196, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.69)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 2.44453394817
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.4445339481694006, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.44)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 0.967589430553
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 0.9675894305526911, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.97)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 0.811147837194
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 11, 't': 14, 'action': None, 'reward': 0.811147837193561, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.81)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: forward, reward: 2.36528924987
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 2.3652892498706186, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 2.37)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: forward, reward: 1.61760467565
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 1.6176046756478732, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.62)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: right, reward: 1.8118271781
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.8118271780965058, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.81)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: right, reward: 0.672583568569
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 0.6725835685689154, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 0.67)
24% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 375
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (4, 6), deadline = 30
Simulating trial. . .
epsilon = 0.0235; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 2.0946491502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.0946491502009073, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.09)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 2.59342354395
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.5934235439479094, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 2.59)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.42982313482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.4298231348175063, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 1.43)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: left, reward: 1.01829449677
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 27, 't': 3, 'action': 'left', 'reward': 1.0182944967664873, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.02)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.75943592773
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.7594359277308418, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.76)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 0.985250566695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 25, 't': 5, 'action': None, 'reward': 0.985250566694621, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 0.99)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: forward, reward: 2.84720289203
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 2.847202892025171, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.85)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: 2.24629788979
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', 'left'), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 2.246297889786927, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', 'left')
Agent followed the waypoint forward. (rewarded 2.25)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 2.65732723529
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.6573272352876725, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.66)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: 2.79808285187
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 2.7980828518693395, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.80)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: right, reward: 2.48912597874
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 2.489125978739075, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.49)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 1.26439921087
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.26439921087125, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.26)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 1.1193884343
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.1193884342974556, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.12)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: forward, reward: 1.68205190161
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.6820519016083797, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.68)
53% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 376
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (4, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0233; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 1.16857921544
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.1685792154354093, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove right instead of left. (rewarded 1.17)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 0.994803307888
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.9948033078880358, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 0.99)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 1.22484214757
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.224842147567088, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.22)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 2.17686333121
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.1768633312115884, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.18)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: 1.30062026341
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 1.3006202634114168, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.30)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: left, reward: 1.41341731798
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.4134173179764207, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.41)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: 2.45268623806
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.4526862380562946, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.45)
72% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 377
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (4, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0231; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0231; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0231; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0231; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 1.45638301716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.4563830171645333, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.46)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 2.90718312269
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 2.9071831226882807, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.91)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 1.73452351858
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.7345235185792123, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 1.73)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: left, reward: -9.18064480993
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -9.18064480993258, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent attempted driving left through a red light. (rewarded -9.18)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 1.96983525631
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.9698352563061523, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.97)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: left, reward: 1.14711904778
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.1471190477795385, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.15)
70% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 378
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (1, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0228; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 2.86922875374
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'right'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.869228753740909, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'right')
Agent followed the waypoint right. (rewarded 2.87)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: left, reward: 1.38093912051
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'right', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.3809391205149386, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'right', None)
Agent drove left instead of right. (rewarded 1.38)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: right, reward: 1.68722773228
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.6872277322827955, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 1.69)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: right, reward: 1.18987880764
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.1898788076358975, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.19)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: forward, reward: 1.15222162723
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.1522216272301078, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.15)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 2.78922283105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.789222831050844, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.79)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: -0.0497781038234
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': -0.04977810382338088, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded -0.05)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: forward, reward: 1.68078865021
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.6807886502123783, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded 1.68)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: 1.49925091726
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.499250917257772, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded 1.50)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 0.933022054575
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.9330220545746963, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 0.93)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.2675396313
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.2675396312971354, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.27)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: 1.08306243486
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 1.083062434856602, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.08)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 1.43704936931
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 1.4370493693091952, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 1.44)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: left, reward: 0.807026873289
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.8070268732885031, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.81)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 1.89029471714
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.8902947171397992, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.89)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: 0.0393346341653
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 0.03933463416531624, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.04)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: -0.576867163007
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': -0.5768671630073552, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded -0.58)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 0.996612125882
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.9966121258822811, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.00)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 1.11167514005
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.1116751400454579, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.11)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 1.30017890456
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 1.3001789045646632, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.30)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 379
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (3, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0226; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0226; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0226; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0226; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 1.56358481382
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'right'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.5635848138203075, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'right')
Agent followed the waypoint right. (rewarded 1.56)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.67816954279
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.678169542794122, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.68)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 1.99957793759
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.9995779375937375, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.00)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 2.4193929188
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.4193929188045487, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.42)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 2.01153912882
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.0115391288244746, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.01)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 0.109308278951
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.1093082789514439, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.11)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: left, reward: 1.64121037681
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.6412103768074011, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.64)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: 1.1148634894
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.1148634894046887, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.11)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 380
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (3, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0224; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0224; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0224; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0224; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 1.03362784811
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.0336278481142278, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.03)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 2.39031864015
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'right'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.3903186401521417, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 2.39)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 2.31955952766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.3195595276597496, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.32)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 2.01413492342
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 2.014134923416758, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.01)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.12112434089
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.121124340891352, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.12)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 2.63141791425
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.631417914247243, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.63)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: 1.0910363549
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.0910363548957362, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.09)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 2.46388539102
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.463885391023121, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.46)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 2.38011765579
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.3801176557911563, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.38)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 1.58173343297
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.5817334329716402, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.58)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 0.947072743384
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'left'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.9470727433836409, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'left')
Agent drove right instead of forward. (rewarded 0.95)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 1.2819431104
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.2819431103999035, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.28)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.77094512597
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.77094512596911, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.77)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: forward, reward: 1.39405977353
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 1.3940597735295686, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.39)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 2.61665625128
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.6166562512798914, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.62)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 1.98555164356
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.985551643561906, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.99)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: left, reward: 0.803342242401
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 0.8033422424007022, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.80)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: left, reward: 1.62042747806
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 1.6204274780616157, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.62)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: 2.23961643884
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': 2.2396164388382274, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.24)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: -10.8712339584
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 6, 't': 19, 'action': 'left', 'reward': -10.871233958410754, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.87)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: 0.746884273798
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 0.746884273797803, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.75)
16% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 381
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (3, 7), deadline = 25
Simulating trial. . .
epsilon = 0.0221; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 0.37289977853
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 0.3728997785296384, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'forward')
Agent drove right instead of left. (rewarded 0.37)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: forward, reward: 0.745495642659
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 0.7454956426587647, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 0.75)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: forward, reward: 1.76376513102
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 1.7637651310195008, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 1.76)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 1.63199778915
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.6319977891463173, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 1.63)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 0.231962591528
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 0.23196259152806287, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.23)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 1.13526517115
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.1352651711484865, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.14)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 2.6332127958
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.633212795796502, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.63)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 0.0124562470027
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.012456247002665677, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 0.01)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 2.4746906826
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 2.4746906825968553, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.47)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 1.62655014528
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.6265501452836468, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.63)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 1.55397330074
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.5539733007353052, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 1.55)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 2.48877880414
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.4887788041384526, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.49)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 2.13171977056
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.131719770560501, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.13)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: left, reward: 2.5090199452
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 2.509019945199856, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.51)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: 0.526692198072
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', 'left'), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 0.5266921980724274, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', 'left')
Agent drove left instead of forward. (rewarded 0.53)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: 1.46271238135
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 1.4627123813489256, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent drove forward instead of right. (rewarded 1.46)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 0.837497205224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 0.8374972052235532, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.84)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 2.56625749313
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 2.566257493131103, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.57)
28% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 382
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (3, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0219; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 2.76864232289
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.7686423228881907, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 2.77)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 0.0452227693572
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.0452227693571875, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', 'left')
Agent drove right instead of forward. (rewarded 0.05)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 1.95411240083
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.9541124008319295, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.95)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 2.28607954778
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.286079547778618, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.29)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 1.72042024592
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.720420245924746, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.72)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 1.14201401078
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.1420140107807735, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.14)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 1.33090665214
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.3309066521403965, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove right instead of left. (rewarded 1.33)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.96505907194
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.9650590719402505, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.97)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 0.971296723229
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.9712967232288616, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.97)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 2.74619396196
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.7461939619645985, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.75)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: left, reward: 0.174469052542
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 0.1744690525418574, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent drove left instead of right. (rewarded 0.17)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: left, reward: 0.774797875724
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 0.7747978757236977, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent drove left instead of right. (rewarded 0.77)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.00023453515
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.000234535147509, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.00)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: forward, reward: 1.04351261007
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.0435126100689955, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.04)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.81214548989
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.812145489887071, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.81)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 0.695855989056
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.6958559890560327, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.70)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: 1.67691760434
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 1.676917604340138, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 1.68)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: 1.39008048661
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 1.3900804866052687, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.39)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.60326059316
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.6032605931573949, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.60)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 0.479243799413
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.4792437994129448, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.48)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 383
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (4, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0217; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 1.8998863805
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'right'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.8998863805039823, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'right')
Agent properly idled at a red light. (rewarded 1.90)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 1.55179289548
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.5517928954788076, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.55)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 1.1133207741
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.1133207740974207, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.11)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 2.80053194455
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 2.800531944551091, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.80)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.26906625064
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.2690662506392183, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.27)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.92814390201
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.9281439020062496, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.93)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.19185185703
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.1918518570270555, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.19)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 0.947307080788
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 0.947307080787724, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.95)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: 0.26131622451
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 0.261316224509745, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', None)
Agent drove left instead of forward. (rewarded 0.26)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: 2.11844095336
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 2.118440953362726, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.12)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.24409399768
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.244093997679617, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.24)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: 1.35587230743
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.3558723074290056, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove left instead of forward. (rewarded 1.36)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 1.37590622714
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.3759062271446683, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.38)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.45093552304
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 2.450935523038283, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.45)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 1.37753912243
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'right'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 1.3775391224327038, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 1.38)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 384
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (7, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0215; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 2.01850669566
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'right'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.0185066956613342, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'right')
Agent followed the waypoint right. (rewarded 2.02)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 1.4792951623
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.4792951623031878, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.48)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.86567958933
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.8656795893297449, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.87)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.94676879083
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.9467687908292595, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.95)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.25815126893
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.258151268930747, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.26)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 1.45850167109
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.45850167108707, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 1.46)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: 0.331343120035
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'left'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 0.33134312003476374, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'left')
Agent drove forward instead of left. (rewarded 0.33)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: 2.45236198586
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 2.4523619858644095, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.45)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 2.38842316328
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 2.388423163275231, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.39)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 2.06239143193
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 2.062391431934949, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.06)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: 1.76093887135
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 1.7609388713508833, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove left instead of forward. (rewarded 1.76)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 1.85032822889
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.850328228891082, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.85)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: None, reward: 1.72023756048
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.7202375604770805, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.72)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 1.73097188734
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.7309718873408713, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.73)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 0.36492607934
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'right'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 0.36492607934048615, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'right')
Agent drove right instead of left. (rewarded 0.36)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: 1.65250531853
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.6525053185311112, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.65)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 0.813831616876
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 0.81383161687637, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.81)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 2.29423649443
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 2.294236494434699, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.29)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 1.5941385667
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 1.5941385666958998, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 1.59)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 0.778473692168
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 19, 'action': None, 'reward': 0.7784736921684043, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.78)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 0.890386403798
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.8903864037983267, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.89)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: left, reward: 1.86582226934
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 1.865822269343908, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.87)
12% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 385
\-------------------------
Environment.reset(): Trial set up with start = (5, 5), destination = (6, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0213; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0213; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0213; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0213; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: left, reward: 1.79391012687
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.793910126874108, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 1.79)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 2.09826080062
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.098260800619462, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.10)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 2.84276732708
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.842767327079419, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.84)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 1.92316615676
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.9231661567563492, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.92)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 1.01961791613
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.0196179161330576, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.02)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: left, reward: 2.90251274856
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.90251274855936, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.90)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.94603277751
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.9460327775080613, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.95)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 1.16178305784
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.1617830578443433, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.16)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 1.62548918013
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.6254891801306002, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.63)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: forward, reward: 0.816749154005
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.8167491540050853, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent followed the waypoint forward. (rewarded 0.82)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 386
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (1, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0211; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0211; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0211; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0211; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: right, reward: 1.68232753335
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.6823275333466376, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 1.68)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 2.75033816342
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.750338163419828, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.75)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 2.23150752871
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.2315075287135406, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.23)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 1.4413559678
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.441355967798754, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.44)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: forward, reward: 1.25734995863
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.2573499586317785, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove forward instead of left. (rewarded 1.26)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: left, reward: 1.1779616554
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.1779616554014845, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.18)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 2.07695312088
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.07695312088177, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.08)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 2.43283602035
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'right'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.4328360203502326, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 2.43)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 1.4998092068
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.4998092067989144, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.50)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: -0.0683722677192
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': -0.06837226771915272, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', None)
Agent drove right instead of forward. (rewarded -0.07)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 1.41748364644
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.4174836464429137, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.42)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 1.18441883564
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.184418835644141, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.18)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 2.59924788075
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.5992478807515687, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.60)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: 2.31626847269
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 2.3162684726938028, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.32)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 1.60851989157
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.608519891567761, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.61)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 1.15784225594
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 1.1578422559415342, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.16)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 0.703722546002
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.7037225460018828, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 0.70)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.02424413387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 2.02424413387327, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.02)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.39564944306
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.395649443062454, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.40)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: left, reward: 1.55514125397
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 1.555141253966587, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.56)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 387
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (3, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0209; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 1.66451423978
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.6645142397839383, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.66)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 2.0928816419
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.0928816419007465, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.09)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 1.08395712485
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.0839571248513578, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.08)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: 2.65416930911
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.6541693091114356, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 2.65)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.04109392848
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.0410939284849392, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.04)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: forward, reward: 0.958344144209
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 0.9583441442093912, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.96)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 2.37180443254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.371804432536787, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.37)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: right, reward: 1.18833834777
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.1883383477665892, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 1.19)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 2.69964248783
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.6996424878255247, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.70)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: left, reward: 0.984095516963
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 0.9840955169628831, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.98)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: left, reward: 2.01680479659
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 2.0168047965931173, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.02)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 388
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (2, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0207; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: forward, reward: 0.502481225691
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 0.5024812256914257, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', None)
Agent drove forward instead of left. (rewarded 0.50)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: 1.56619828035
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 1.5661982803535495, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.57)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: 2.73898521101
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 2.73898521100874, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.74)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.24390432895
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.2439043289492213, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.24)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: 1.16809294804
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 1.1680929480362834, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.17)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: forward, reward: 0.0758722612267
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 0.07587226122666968, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 0.08)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 2.1873571095
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.187357109497543, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.19)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.379605498
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.379605497999597, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.38)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 0.21369259479
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.21369259478965186, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.21)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 2.49175442385
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.4917544238536546, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.49)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 2.19874470795
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.1987447079482356, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.20)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: forward, reward: 1.07291009349
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.0729100934931748, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove forward instead of left. (rewarded 1.07)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: 1.24831495714
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 12, 'action': 'left', 'reward': 1.2483149571425762, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.25)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: -40.0545703892
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', 'right'), 'deadline': 12, 't': 13, 'action': 'left', 'reward': -40.054570389205246, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.05)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.2041219089
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'right'), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.2041219088992658, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 1.20)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.38459487105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.3845948710530733, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.38)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.88596075917
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.885960759172515, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.89)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 0.819082482358
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 0.8190824823577421, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove forward instead of left. (rewarded 0.82)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: 0.145887218044
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'forward'), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': 0.14588721804375804, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.15)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: left, reward: 1.06386778956
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 1.0638677895596158, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.06)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 2.13278704737
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 2.132787047365192, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.13)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: left, reward: 1.7936118311
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 1.7936118311031881, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.79)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.85326504299
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 3, 't': 22, 'action': None, 'reward': 1.8532650429875743, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.85)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 0.69144257024
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 2, 't': 23, 'action': None, 'reward': 0.6914425702395204, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.69)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.56107253924
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.561072539242202, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.56)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 389
\-------------------------
Environment.reset(): Trial set up with start = (5, 7), destination = (8, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0204; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0204; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0204; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.97622686665
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.9762268666452245, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.98)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: right, reward: 2.0884090386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.0884090386028755, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.09)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 1.20536667921
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.2053666792060038, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.21)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 2.13977807851
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.139778078509301, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.14)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 2.01354120944
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.013541209439474, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.01)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 2.44288399649
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 2.4428839964881517, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.44)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.86086780268
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.8608678026817758, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.86)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: left, reward: 1.37340949288
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 1.3734094928837515, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.37)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 2.69976780579
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.6997678057915113, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.70)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: left, reward: 1.81085111967
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.8108511196702235, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove left instead of right. (rewarded 1.81)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 0.983940460582
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.9839404605824322, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.98)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: 2.51985895016
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 2.5198589501619653, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.52)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 2.64454196231
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 2.644541962313839, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.64)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 1.99655500596
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.9965550059606578, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.00)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 0.717900028962
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 0.7179000289622863, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.72)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 2.14668688377
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.1466868837739073, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.15)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: forward, reward: 1.36913880283
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 1.369138802830032, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.37)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.0068732156
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 8, 't': 17, 'action': None, 'reward': 1.0068732155989732, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.01)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.55774216395
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.5577421639539866, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.56)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: left, reward: -0.19118392306
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 6, 't': 19, 'action': 'left', 'reward': -0.19118392306027554, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded -0.19)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 0.500724890068
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 0.5007248900684333, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 0.50)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: 1.91886622621
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 1.9188662262052723, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.92)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 1.86347484992
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'right'), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 1.8634748499214804, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 1.86)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 0.610063258764
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 2, 't': 23, 'action': 'right', 'reward': 0.6100632587638997, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded 0.61)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 0.518102188838
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 1, 't': 24, 'action': 'right', 'reward': 0.5181021888377897, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.52)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 390
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (4, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0202; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0202; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0202; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: forward, reward: 1.54420728784
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.5442072878396642, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove forward instead of left. (rewarded 1.54)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 1.11024361442
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'right'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.110243614419467, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'right')
Agent drove right instead of left. (rewarded 1.11)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 2.71845393794
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.71845393794074, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.72)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 1.7423731948
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.7423731948017493, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.74)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 1.07859347354
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.0785934735382776, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.08)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 1.10741945852
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'right'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.1074194585210488, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'right')
Agent followed the waypoint right. (rewarded 1.11)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 2.76626460353
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 2.7662646035333545, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.77)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 2.60554851071
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.6055485107089167, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.61)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 0.789978080581
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'forward'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.7899780805814317, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.79)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: forward, reward: 0.871602307247
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.8716023072467318, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove forward instead of left. (rewarded 0.87)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: forward, reward: -0.0258588328831
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': -0.02585883288307722, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded -0.03)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 2.03073814426
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.030738144262858, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.03)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 1.66356392394
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.6635639239398055, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.66)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 1.97770168756
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.9777016875635827, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', None)
Agent properly idled at a red light. (rewarded 1.98)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 1.83556562893
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.8355656289329287, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.84)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 0.772271576977
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.7722715769774735, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.77)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: left, reward: 0.971778777732
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 0.9717787777316285, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 0.97)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: left, reward: 1.03195945129
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 1.0319594512873387, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent drove left instead of right. (rewarded 1.03)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: 0.0117211121476
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 0.011721112147577095, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent drove forward instead of right. (rewarded 0.01)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 0.648768656876
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.648768656876404, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.65)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 391
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (2, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0200; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 2.94678331607
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.946783316074705, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.95)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 2.28525392558
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.285253925580704, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 2.29)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.69702326692
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.6970232669215668, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 1.70)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.98512756839
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.9851275683875418, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.99)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 2.37565694761
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.3756569476073954, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.38)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.78217723502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.7821772350193934, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.78)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: forward, reward: 0.444206260043
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 0.44420626004272457, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.44)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: -5.14540804962
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': None, 'reward': -5.145408049623899, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.15)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: 2.42355312963
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 2.4235531296308266, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.42)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.74116152667
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.741161526674871, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.74)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 0.786117571368
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.7861175713684562, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.79)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 1.57966392857
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.579663928566689, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.58)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 1.66480809134
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.664808091343873, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.66)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: left, reward: 2.48876940496
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 2.488769404955847, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.49)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 1.00032346735
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.0003234673495953, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 1.00)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: 1.43052202486
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 1.4305220248623223, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent followed the waypoint forward. (rewarded 1.43)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 1.02902390921
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.0290239092054125, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.03)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: forward, reward: 0.950321639532
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.9503216395324192, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.95)
10% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 392
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (2, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0198; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0198; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0198; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0198; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 2.31830818105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.3183081810545847, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.32)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 1.06343828999
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.0634382899927666, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.06)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 0.188624260441
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.18862426044054337, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.19)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 2.71809031755
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.7180903175473494, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.72)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 2.88962201141
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.889622011408903, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.89)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 0.434097187007
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'right'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.4340971870070722, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'right')
Agent drove right instead of left. (rewarded 0.43)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 1.89387960933
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.8938796093334511, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.89)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 1.47903173087
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.4790317308733132, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.48)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 2.5336162205
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'forward'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 2.533616220499616, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 2.53)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 1.43910284091
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.439102840909457, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 1.44)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 1.43066059187
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.4306605918736364, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.43)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: 0.16053508568
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 0.1605350856796648, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.16)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 0.780028367167
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.7800283671666599, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 0.78)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 1.1479530652
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.1479530651962606, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.15)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 2.1567001224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.15670012239857, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.16)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 1.84991634507
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'right', None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.8499163450720322, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', None)
Agent followed the waypoint right. (rewarded 1.85)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: -0.249939635591
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': -0.24993963559083499, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded -0.25)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 0.81973918424
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'left'), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.8197391842395468, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'left')
Agent drove right instead of left. (rewarded 0.82)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 1.00900813361
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 1.009008133611308, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.01)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 1.74588474972
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 1.7458847497159895, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 1.75)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 393
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (5, 2), deadline = 30
Simulating trial. . .
epsilon = 0.0196; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0196; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0196; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: right, reward: 0.414514970642
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 0.4145149706417208, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'left')
Agent drove right instead of left. (rewarded 0.41)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 2.60735939745
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 2.607359397451085, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.61)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 2.23904580868
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.2390458086754754, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.24)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.58235177331
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.5823517733100334, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.58)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.09960281112
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.0996028111190526, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.10)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 2.92680710179
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.926807101789346, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.93)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: forward, reward: 1.1538717462
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 1.1538717461993597, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.15)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 1.17425166509
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.1742516650884762, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.17)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 2.88629865629
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.8862986562873747, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.89)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 0.00743548384457
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'right'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 0.007435483844570934, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'right')
Agent drove right instead of forward. (rewarded 0.01)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 2.37700649254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.3770064925407333, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.38)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.78909280434
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.7890928043443242, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.79)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 0.868328587538
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 18, 't': 12, 'action': None, 'reward': 0.8683285875381108, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.87)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: left, reward: 2.48239882315
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 2.4823988231492473, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 2.48)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: 1.03451668093
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': 1.0345166809285251, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent drove forward instead of right. (rewarded 1.03)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 0.968600012243
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 0.9686000122434095, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 0.97)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 1.97222897472
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 1.9722289747184483, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 1.97)
43% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 394
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (3, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0194; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0194; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0194; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0194; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.3525903471
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.3525903471042375, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.35)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.49481915571
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'right'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.4948191557054231, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.49)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: right, reward: 1.13589993947
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.1358999394671232, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.14)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: 2.55106636371
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.5510663637135575, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent followed the waypoint forward. (rewarded 2.55)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 2.82873020723
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.828730207229272, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.83)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 2.68896907518
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.6889690751815634, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.69)
70% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 395
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (6, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0193; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0193; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 1.88628980691
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.88628980691151, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.89)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: left, reward: 1.44413262836
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.444132628361894, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.44)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: left, reward: 0.290729624007
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 0.29072962400705504, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.29)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 1.77067470412
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.7706747041231956, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.77)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 2.55511715254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.555117152535368, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.56)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 1.94397359344
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.9439735934444868, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.94)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 1.4451658944
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.4451658944041177, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.45)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: forward, reward: 0.718646464507
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 0.7186464645074082, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', None)
Agent drove forward instead of left. (rewarded 0.72)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: left, reward: 1.19545732491
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.1954573249129925, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.20)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 1.8937896237
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.8937896236965874, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.89)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 1.08591195629
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.0859119562909838, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.09)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: right, reward: 0.431167054855
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.4311670548553881, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 0.43)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 2.38541194777
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 2.385411947768319, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.39)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 0.712925695584
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 0.7129256955835064, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.71)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: 2.08235935269
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 2.082359352693456, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.08)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: left, reward: 0.460296607043
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 0.46029660704273545, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.46)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 1.89443459041
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.8944345904115012, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.89)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: 0.796713827286
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.7967138272859798, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.80)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: right, reward: 1.64813080187
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 1.6481308018737217, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.65)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 2.00131833878
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 2.0013183387805435, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 2.00)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 396
\-------------------------
Environment.reset(): Trial set up with start = (7, 4), destination = (4, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0191; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: forward, reward: 0.704447994507
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 0.704447994507446, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent drove forward instead of right. (rewarded 0.70)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: right, reward: 2.32528663203
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.3252866320327024, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.33)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 1.3739670494
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.3739670494019973, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.37)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 1.31151425333
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.3115142533338753, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.31)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: left, reward: -0.0348054976282
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -0.03480549762822227, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded -0.03)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: None, reward: -5.36750337037
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', 'right', None), 'deadline': 15, 't': 5, 'action': None, 'reward': -5.367503370366009, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.37)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: 2.15444344831
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 2.1544434483081103, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.15)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.56120871278
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.5612087127811327, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.56)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.5887030702
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.588703070198474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.59)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: forward, reward: 1.62940434405
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.6294043440497896, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 1.63)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.51900452583
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.5190045258261589, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.52)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.99817505211
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.998175052112227, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.00)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.68778313669
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.6877831366944784, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.69)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: forward, reward: 1.80070142705
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.800701427049066, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.80)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 397
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (2, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0189; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 1.48275256896
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.4827525689600964, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'left')
Agent followed the waypoint right. (rewarded 1.48)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 2.2949365343
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', 'right'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.294936534297399, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', 'right')
Agent properly idled at a red light. (rewarded 2.29)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 2.34810158862
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.348101588623992, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.35)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 2.27180193165
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.27180193164832, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.27)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 2.93487479782
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.9348747978224603, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.93)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 1.91049398796
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.9104939879632197, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.91)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: left, reward: 1.55073063248
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.5507306324815424, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 1.55)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 1.82374587148
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.8237458714814652, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.82)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 2.28156944421
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'left'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.2815694442137384, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'left')
Agent followed the waypoint right. (rewarded 2.28)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 0.84474849799
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.8447484979899749, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent followed the waypoint forward. (rewarded 0.84)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.75521051055
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.755210510549867, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.76)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.23389388517
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.2338938851716723, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.23)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 2.52998959129
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.529989591290276, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.53)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: left, reward: 2.5538223157
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 2.5538223157002267, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.55)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 398
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (3, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0187; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0187; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0187; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0187; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: forward, reward: 0.204208906299
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 0.20420890629939525, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.20)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: right, reward: 1.78060182198
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.7806018219805033, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 1.78)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 2.45112173193
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.451121731925281, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.45)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 2.53080172804
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.530801728039661, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.53)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 2.42782427352
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.4278242735170217, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.43)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: 1.71572874967
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.715728749665095, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.72)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 1.24084354526
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.2408435452592381, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.24)
65% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 399
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (1, 2), deadline = 30
Simulating trial. . .
epsilon = 0.0185; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 2.38580200677
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'right'), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.385802006773705, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 2.39)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 2.79732992911
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.7973299291099916, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.80)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 2.65630893725
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.656308937250747, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.66)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: left, reward: 1.9385914378
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 27, 't': 3, 'action': 'left', 'reward': 1.9385914378003923, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.94)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: None, reward: 1.95609682494
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.9560968249394526, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.96)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: left, reward: 1.2304067193
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 1.230406719298985, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.23)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 1.09353964027
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.0935396402706443, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.09)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 1.4526999417
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.4526999416956103, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.45)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 1.59577138331
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.5957713833144689, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.60)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: 1.15639941543
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.1563994154320645, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.16)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 1.96140739667
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.9614073966694754, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.96)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 2.10589907176
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 2.105899071758968, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.11)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.40877666742
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 1.4087766674171538, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.41)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 1.25459146191
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 13, 'action': None, 'reward': 1.254591461908901, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.25)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 2.3654937037
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 16, 't': 14, 'action': None, 'reward': 2.365493703704253, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.37)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: forward, reward: 2.20237486531
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': 2.2023748653053055, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.20)
47% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 400
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (2, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0183; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 2.88589850287
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.8858985028744293, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.89)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 1.04023675605
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.040236756046431, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.04)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 1.8036175187
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.8036175186961454, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.80)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 2.2278535672
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 2.2278535672027386, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.23)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 1.04188584588
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.0418858458764262, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.04)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.8916199353
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.8916199353022325, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.89)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 1.58629761648
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.5862976164786515, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 1.59)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.11707742574
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.117077425742921, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.12)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.59065163835
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.5906516383481906, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.59)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: left, reward: 1.1779673681
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.1779673680958023, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.18)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: 0.90876445817
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 0.9087644581700969, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 0.91)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 401
\-------------------------
Environment.reset(): Trial set up with start = (3, 2), destination = (7, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0181; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 1.47725882241
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.4772588224132788, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.48)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 2.17487528185
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.1748752818529073, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.17)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: left, reward: 1.75429711099
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.7542971109907604, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.75)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 2.03690635718
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 2.036906357179079, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.04)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.23746234107
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.2374623410655268, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.24)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.1364412605
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.1364412604986927, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.14)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 0.952658109937
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 0.9526581099368459, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.95)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 2.59502927669
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.595029276688837, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 2.60)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 2.20345924271
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.203459242714297, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.20)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 2.06811356624
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.068113566237112, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.07)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: right, reward: 0.958915452898
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.9589154528981979, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', None)
Agent drove right instead of forward. (rewarded 0.96)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: left, reward: 1.34404271109
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 1.344042711089379, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.34)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: left, reward: 0.908724532662
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 0.9087245326617197, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 0.91)
35% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 402
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (6, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0180; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0180; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0180; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 2.15604146524
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.156041465241304, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.16)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 2.8606039631
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.8606039630965236, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.86)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 1.75445174651
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.7544517465127063, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.75)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 2.24443572207
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.244435722074015, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.24)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 2.31212985511
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.3121298551135174, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.31)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: forward, reward: 2.17141467101
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 2.171414671005696, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.17)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 1.93396994402
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.9339699440218794, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.93)
65% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 403
\-------------------------
Environment.reset(): Trial set up with start = (5, 2), destination = (1, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0178; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 1.38763195427
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', 'right'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.387631954265932, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', 'right')
Agent followed the waypoint forward. (rewarded 1.39)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 1.67620955285
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.6762095528495247, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 1.68)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: 1.39763941174
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 1.3976394117444586, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.40)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: 2.00988576935
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'right'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.009885769351656, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'right')
Agent followed the waypoint forward. (rewarded 2.01)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 1.70065401048
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.7006540104834629, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.70)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 1.27733111181
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.277331111806759, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.28)
76% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 404
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (1, 2), deadline = 30
Simulating trial. . .
epsilon = 0.0176; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 2.68774167828
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.6877416782794104, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.69)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 1.82190866369
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.8219086636890862, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.82)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 2.77924782585
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.7792478258458813, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.78)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 1.16635931478
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.1663593147832823, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.17)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: forward, reward: 2.50353907613
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 2.5035390761284253, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.50)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 2.91365971015
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 2.913659710145494, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.91)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: left, reward: 1.83344944182
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 1.833449441822967, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.83)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: right, reward: 2.54030580061
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 2.540305800614732, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.54)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 1.39561910656
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.3956191065645422, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.40)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 2.64610942515
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 2.6461094251491097, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.65)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: 1.32591912276
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 1.3259191227572915, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent followed the waypoint left. (rewarded 1.33)
63% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 405
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (8, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0174; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0174; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0174; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 1.60467051656
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.6046705165649495, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', 'forward')
Agent followed the waypoint right. (rewarded 1.60)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 1.29597975793
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 1.295979757928713, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.30)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 1.81065738081
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.8106573808104502, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.81)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 2.50533932123
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 2.505339321234298, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.51)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: forward, reward: 2.52125146417
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.521251464172056, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.52)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 0.294768608223
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.29476860822340967, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 0.29)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: left, reward: 1.08711654712
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.0871165471216082, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.09)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 2.81100989932
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.8110098993201458, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.81)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: forward, reward: -10.61191234
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -10.611912340041366, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.61)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 2.16494394946
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.1649439494647575, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.16)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: -4.93482736941
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'right', 'forward'), 'deadline': 15, 't': 10, 'action': None, 'reward': -4.934827369406952, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.93)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: left, reward: 0.939279359129
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 0.9392793591287938, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 0.94)
52% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 406
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (3, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0172; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 1.92770975448
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.9277097544845145, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.93)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 1.79412108985
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.794121089847013, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.79)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: left, reward: 2.30271652509
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 2.3027165250936212, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.30)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 2.54682810407
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.5468281040733705, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.55)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.94944007962
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 1.9494400796232083, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.95)
80% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 407
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (2, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0171; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 1.51239656034
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.5123965603352452, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'left')
Agent drove right instead of left. (rewarded 1.51)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 2.30317796386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.3031779638620855, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.30)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 2.85431104618
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.854311046184736, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.85)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.27156862835
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.2715686283480998, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.27)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 2.91844047915
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.9184404791485443, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.92)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: 2.8974403762
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.8974403761965015, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.90)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: 1.11622065608
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.1162206560814842, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove forward instead of left. (rewarded 1.12)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.98941059834
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.9894105983358321, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.99)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.63042790754
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.6304279075409622, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.63)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: 1.23572777899
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.2357277789936651, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.24)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.32891048004
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.3289104800396196, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 2.33)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 0.891471154391
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 9, 't': 11, 'action': None, 'reward': 0.8914711543905589, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.89)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: right, reward: 0.748367414837
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.7483674148365727, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.75)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 2.17840237808
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 2.1784023780799933, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.18)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 1.49138629253
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'right'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.4913862925334918, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'right')
Agent properly idled at a red light. (rewarded 1.49)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 1.0345472215
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.0345472214981604, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent drove right instead of left. (rewarded 1.03)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 1.14428614846
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'right'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.1442861484642934, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'right')
Agent followed the waypoint right. (rewarded 1.14)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 2.0645800588
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 2.0645800588036654, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 2.06)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 0.395569799421
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.3955697994209, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.40)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 0.267517209535
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 0.26751720953472957, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.27)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 408
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (4, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0169; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0169; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0169; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 1.64949550461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.649495504606125, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.65)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 1.60557688978
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.605576889783402, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.61)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 1.36400150823
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.364001508226782, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.36)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 2.18832534078
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.188325340780475, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.19)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 1.30878988735
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.3087898873450847, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.31)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 2.01062198411
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.010621984111128, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.01)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: left, reward: 1.79146630337
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.79146630336957, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.79)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 1.60877824405
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.608778244048909, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.61)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 0.855818740648
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 0.8558187406481019, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.86)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 1.35538454761
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.3553845476056965, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.36)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 1.3749170107
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.3749170107036164, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.37)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 409
\-------------------------
Environment.reset(): Trial set up with start = (2, 7), destination = (7, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0167; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0167; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0167; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0167; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: right, reward: 1.41427654706
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.4142765470608485, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.41)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 1.02299665907
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.022996659071362, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.02)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 1.52139918295
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.5213991829482467, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.52)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 1.66931344899
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.669313448989175, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.67)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 2.36517124848
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.365171248476166, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.37)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.08095750143
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.080957501433403, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.08)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 1.47539681184
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.4753968118390972, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.48)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 0.916414620681
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'left'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.9164146206808343, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'left')
Agent drove right instead of left. (rewarded 0.92)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 2.19234126456
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.192341264561869, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.19)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 2.03424308656
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.0342430865584653, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 2.03)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 2.33243763622
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.332437636216415, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.33)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 2.01463677797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.0146367779731227, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.01)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 2.66500797012
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 2.6650079701151115, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 2.67)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: 1.13710533053
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.1371053305309418, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.14)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 410
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (1, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0166; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0166; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 2.63086356527
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.63086356526829, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.63)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 2.16506543911
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.1650654391058044, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.17)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 2.44758469544
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.4475846954363942, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.45)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: 1.33234968506
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.332349685055358, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.33)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 2.94314818123
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 2.9431481812265208, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.94)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 1.74393660583
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.7439366058263799, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.74)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 2.49436635369
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.4943663536932688, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.49)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: 2.37412475053
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 2.3741247505293064, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.37)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: 0.914366268038
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 0.914366268037959, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 0.91)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 1.20927294103
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.2092729410252232, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.21)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: 2.71651190202
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 2.716511902019211, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 2.72)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 1.20438736476
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.204387364755136, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.20)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 1.94101483303
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.9410148330298205, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.94)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 0.261844172608
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 0.26184417260815995, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.26)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 0.0385060214989
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 0.03850602149893645, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent drove right instead of left. (rewarded 0.04)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 0.690639985937
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 0.69063998593681, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.69)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: 2.46321127431
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 2.4632112743126613, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.46)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: forward, reward: 1.82332825871
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 1.8233282587081152, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.82)
28% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 411
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (1, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0164; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 1.80871510575
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.8087151057460737, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.81)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: 2.41860877935
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 2.418608779348945, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.42)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 2.61371105039
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 2.6137110503903145, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.61)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.89271397657
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.892713976565052, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.89)
80% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 412
\-------------------------
Environment.reset(): Trial set up with start = (4, 3), destination = (1, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0162; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 2.47437995041
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 2.474379950408636, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 2.47)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 2.78220691937
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.7822069193749273, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.78)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 1.06771160798
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.0677116079774949, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.07)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 1.40028326896
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.4002832689586433, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.40)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: 2.72535714126
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.725357141259915, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.73)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 2.3035629157
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.3035629157039796, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.30)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: left, reward: 2.17152461497
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 2.171524614972794, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.17)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 1.69590044327
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.695900443272576, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.70)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 1.16477039693
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.1647703969297767, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.16)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 2.8098603317
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.809860331697322, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.81)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 2.77189881746
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.771898817464638, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.77)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: forward, reward: 2.11678979436
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 2.1167897943590086, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.12)
52% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 413
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (8, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0161; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 2.54429135771
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.5442913577073965, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.54)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 1.31124886982
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.311248869819908, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 1.31)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 2.34526594493
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.3452659449301025, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.35)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 1.4447724932
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.44477249319928, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.44)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 2.46832188463
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.468321884634525, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.47)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: forward, reward: 0.466796307718
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 0.46679630771751945, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.47)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 1.16929123779
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.169291237787408, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.17)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: 1.40002077917
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.4000207791742572, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.40)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: 2.70787664587
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 2.707876645865973, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent followed the waypoint forward. (rewarded 2.71)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 0.0337087112555
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.033708711255512114, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', None)
Agent drove forward instead of left. (rewarded 0.03)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 2.19591053377
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.195910533766327, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.20)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: left, reward: 1.86519679394
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 1.865196793943752, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.87)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 1.26773453958
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.267734539581041, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.27)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: forward, reward: 1.4448128207
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.4448128206975914, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.44)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: forward, reward: -40.1102637707
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', 'left'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': -40.11026377067185, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.11)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 0.834746804258
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.8347468042578423, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.83)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: left, reward: 0.618204566008
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 0.6182045660078141, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.62)
15% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 414
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (8, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0159; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0159; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0159; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0159; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: forward, reward: 0.352146287109
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'forward'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 0.3521462871091534, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'forward')
Agent drove forward instead of left. (rewarded 0.35)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: forward, reward: 1.72419766766
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 1.72419766765743, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.72)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 1.40255742725
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.402557427252819, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.40)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 1.12989764318
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.1298976431834709, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.13)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 1.62891071719
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.628910717193402, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.63)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: left, reward: 0.939700531637
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 0.9397005316368969, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.94)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 1.68220840746
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.68220840746207, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.68)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 1.38078345725
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.3807834572518007, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.38)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 1.09632165095
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.0963216509491462, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.10)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: 2.54542147
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.5454214700033506, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.55)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: 2.45223330427
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.4522333042672697, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.45)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: 2.48268947257
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 2.482689472569101, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.48)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: 1.50381429558
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 13, 't': 12, 'action': 'left', 'reward': 1.50381429558277, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.50)
48% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 415
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (5, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0158; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0158; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0158; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 1.11240146406
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.1124014640561117, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.11)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 0.265625145642
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.265625145641761, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 0.27)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 2.9245251497
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.924525149703779, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.92)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 1.09623582313
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.096235823134972, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.10)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 2.60926662715
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.609266627152427, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.61)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: forward, reward: 0.953638214162
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 0.9536382141619548, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.95)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: 1.04617418653
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.0461741865260656, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.05)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.56291816181
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.5629181618100922, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.56)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.69187164561
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.6918716456119167, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.69)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.31083696005
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.3108369600471104, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.31)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.34066053681
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.340660536807711, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.34)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: 1.92237940152
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.922379401521227, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.92)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: 1.32178223049
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 1.3217822304886628, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.32)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 2.07799630938
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 2.0779963093844627, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.08)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 1.69887935837
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.6988793583737523, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.70)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 1.88766820539
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.8876682053912197, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.89)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 2.29686749002
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 2.2968674900246677, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.30)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: right, reward: 0.362554433467
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.3625544334673707, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.36)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: right, reward: 0.624885365471
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 0.6248853654705331, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 0.62)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: right, reward: 1.32765166641
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'forward'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 1.3276516664129792, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'right', None, 'forward')
Agent followed the waypoint right. (rewarded 1.33)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 416
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (5, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0156; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 1.57909183287
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'right'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.5790918328679164, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'right')
Agent properly idled at a red light. (rewarded 1.58)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 1.6062581902
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.6062581901996753, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.61)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 2.90795210378
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.907952103782166, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.91)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 1.98394801194
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.9839480119350046, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.98)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 2.00229032269
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.0022903226942805, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.00)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 1.70388314448
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.7038831444767046, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.70)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: left, reward: 1.83026147432
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.8302614743172851, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.83)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: 1.00568612156
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.0056861215573902, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.01)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 2.01208665656
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.0120866565577193, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.01)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 1.97160474022
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.9716047402190509, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.97)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 2.44473856508
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 2.444738565079504, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.44)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 0.923492013428
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 14, 't': 11, 'action': None, 'reward': 0.9234920134278972, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.92)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 2.05553411486
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.0555341148611284, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.06)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: 2.62789751786
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 2.627897517860296, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.63)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: forward, reward: 1.11547638222
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'forward'), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 1.1154763822151454, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'forward')
Agent drove forward instead of left. (rewarded 1.12)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: left, reward: 2.48114849717
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 2.4811484971727165, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.48)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 1.94705462358
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'right'), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.9470546235802908, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 1.95)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 2.39705603805
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 8, 't': 17, 'action': None, 'reward': 2.3970560380515877, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.40)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 1.23517075735
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.2351707573528063, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.24)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 0.475968968529
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 6, 't': 19, 'action': None, 'reward': 0.4759689685291635, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 0.48)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 1.28081742715
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.280817427154515, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.28)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: left, reward: 0.826685171107
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 0.8266851711065206, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.83)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 0.431237870715
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.43123787071466313, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.43)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 1.5206262233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.5206262232965762, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.52)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 1.60991511956
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.6099151195614507, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.61)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 417
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (4, 5), deadline = 35
Simulating trial. . .
epsilon = 0.0155; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: left, reward: 1.0356993027
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 35, 't': 0, 'action': 'left', 'reward': 1.0356993027036794, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.04)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 2.12057227535
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 34, 't': 1, 'action': 'forward', 'reward': 2.120572275352742, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.12)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.28926608142
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 33, 't': 2, 'action': None, 'reward': 1.2892660814202241, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.29)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 0.682806231865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 32, 't': 3, 'action': 'right', 'reward': 0.6828062318654542, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 0.68)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 2.11828515185
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 31, 't': 4, 'action': None, 'reward': 2.1182851518501, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.12)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: left, reward: 1.71766241789
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 30, 't': 5, 'action': 'left', 'reward': 1.717662417890429, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.72)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 2.04529486041
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 29, 't': 6, 'action': 'forward', 'reward': 2.045294860406681, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.05)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 1.4995091887
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 28, 't': 7, 'action': 'right', 'reward': 1.4995091886997243, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.50)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 0.353830357524
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', 'forward'), 'deadline': 27, 't': 8, 'action': 'right', 'reward': 0.3538303575243408, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.35)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 1.07141191211
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 26, 't': 9, 'action': None, 'reward': 1.0714119121106374, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.07)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 2.35649676179
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 25, 't': 10, 'action': None, 'reward': 2.3564967617855994, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.36)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: left, reward: 0.909285759171
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 24, 't': 11, 'action': 'left', 'reward': 0.9092857591707262, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 0.91)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 2.57651634356
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 12, 'action': None, 'reward': 2.5765163435554816, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.58)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 2.57416380433
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 22, 't': 13, 'action': None, 'reward': 2.5741638043329633, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.57)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 1.61348698224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 21, 't': 14, 'action': 'right', 'reward': 1.6134869822401265, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent drove right instead of left. (rewarded 1.61)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.33415734618
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 20, 't': 15, 'action': None, 'reward': 2.334157346178798, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.33)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: right, reward: 2.73640187646
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 19, 't': 16, 'action': 'right', 'reward': 2.736401876457347, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.74)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 2.60767786871
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 18, 't': 17, 'action': 'right', 'reward': 2.6076778687106117, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 2.61)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 1.55989716425
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 17, 't': 18, 'action': 'right', 'reward': 1.559897164248539, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 1.56)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: left, reward: 2.37136840175
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 16, 't': 19, 'action': 'left', 'reward': 2.371368401746812, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.37)
43% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 418
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (4, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0153; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0153; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0153; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 1.80631610356
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.8063161035609514, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 1.81)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.00995514256
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.0099551425600959, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.01)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.60086654622
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.6008665462172542, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.60)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.49234487985
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.492344879852255, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.49)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 2.71920990791
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.719209907910468, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.72)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: left, reward: 1.9909089019
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.9909089019030088, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.99)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: forward, reward: 2.34880106309
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.3488010630886276, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.35)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 2.72780302266
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.7278030226618073, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.73)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 1.46695003359
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.4669500335853538, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.47)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: forward, reward: 1.80707028077
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.807070280768876, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.81)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 419
\-------------------------
Environment.reset(): Trial set up with start = (7, 3), destination = (4, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0151; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0151; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0151; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: right, reward: 2.80436781047
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.8043678104741945, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.80)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 1.14670199278
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.1467019927782354, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.15)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: left, reward: 0.534489445544
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 0.5344894455441157, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.53)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 1.7956059399
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.7956059398992297, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.80)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: left, reward: 0.323330276805
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 0.3233302768045633, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.32)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 1.2796072888
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.2796072887952368, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.28)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 1.77647766474
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.7764776647399632, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.78)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 2.42295244945
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.4229524494464467, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.42)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 2.18235744736
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.182357447360916, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.18)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: 1.24668072511
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.2466807251094993, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.25)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: right, reward: 2.54338752964
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 2.5433875296420574, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.54)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 2.27365041476
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.2736504147586656, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.27)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 1.28427553971
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.2842755397141337, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.28)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: 1.16787294683
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.1678729468345441, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.17)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 420
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (8, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0150; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0150; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.8134380712
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.8134380712006024, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.81)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.16479887317
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.1647988731661894, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.16)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.40241439535
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.4024143953527313, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.40)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.33764634522
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.3376463452152623, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.34)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.92364120134
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.9236412013421798, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.92)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.16280591323
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.1628059132340742, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.16)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: forward, reward: 2.71331460201
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 2.713314602008894, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.71)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: 1.16834174735
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.1683417473548852, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.17)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: right, reward: 1.25427517821
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.2542751782059367, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.25)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 1.82748872827
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.8274887282748267, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.83)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 2.46233292096
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.4623329209605282, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.46)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: forward, reward: 1.23267056971
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.2326705697089428, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent followed the waypoint forward. (rewarded 1.23)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 421
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (2, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0148; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0148; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 2.4583195116
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.4583195115995435, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', 'left')
Agent properly idled at a red light. (rewarded 2.46)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 1.95484202597
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.954842025966667, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', 'left')
Agent properly idled at a red light. (rewarded 1.95)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 2.57715355092
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 2.5771535509200003, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.58)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 1.26568221982
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.265682219824026, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.27)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 2.48223149135
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 2.482231491354888, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.48)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 2.86168096541
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.861680965405857, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.86)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: left, reward: 1.04391319417
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.0439131941731072, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 1.04)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 0.911624293379
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 0.9116242933788061, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.91)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 0.968561531864
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 0.9685615318637515, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.97)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 0.87380101036
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.873801010360463, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 0.87)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 2.64600383274
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.6460038327429833, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.65)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: forward, reward: 0.961874983411
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 0.9618749834107745, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.96)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 2.01193151634
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.0119315163432265, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.01)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.65818337706
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.6581833770598873, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.66)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.88390101627
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.8839010162658079, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.88)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 2.40758673819
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.407586738194974, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.41)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: left, reward: 0.723060822142
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 0.723060822142037, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.72)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 0.998804093935
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', 'left'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 0.9988040939352112, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', 'left')
Agent followed the waypoint right. (rewarded 1.00)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.19465911543
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.1946591154349664, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.19)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 1.25359856537
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 1.2535985653702186, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.25)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 0.220394563574
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 0.22039456357446152, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 0.22)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.91817699274
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 4, 't': 21, 'action': None, 'reward': 1.918176992738706, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.92)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: -0.690187274167
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 3, 't': 22, 'action': 'right', 'reward': -0.6901872741665198, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded -0.69)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 0.963355374567
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 23, 'action': None, 'reward': 0.9633553745669774, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.96)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 1.21556688463
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 1, 't': 24, 'action': 'right', 'reward': 1.215566884631767, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.22)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 422
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (3, 2), deadline = 35
Simulating trial. . .
epsilon = 0.0147; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 0.306006668345
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'right'), 'deadline': 35, 't': 0, 'action': 'right', 'reward': 0.3060066683451905, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'right')
Agent drove right instead of left. (rewarded 0.31)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 2.97786814095
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 34, 't': 1, 'action': 'right', 'reward': 2.9778681409524417, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.98)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 2.31847941176
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 33, 't': 2, 'action': None, 'reward': 2.3184794117586494, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.32)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: forward, reward: 1.20270967473
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 32, 't': 3, 'action': 'forward', 'reward': 1.2027096747286656, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.20)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.67677619466
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 31, 't': 4, 'action': None, 'reward': 1.6767761946576414, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.68)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.60510148094
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 30, 't': 5, 'action': None, 'reward': 1.6051014809411392, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.61)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: forward, reward: 1.36718755291
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 29, 't': 6, 'action': 'forward', 'reward': 1.3671875529137612, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.37)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: left, reward: 0.672816555562
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 28, 't': 7, 'action': 'left', 'reward': 0.6728165555616435, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove left instead of forward. (rewarded 0.67)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: 2.30146129052
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 27, 't': 8, 'action': 'right', 'reward': 2.301461290516764, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.30)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 1.76750378154
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 26, 't': 9, 'action': None, 'reward': 1.7675037815384893, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.77)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 0.142125323951
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 25, 't': 10, 'action': 'forward', 'reward': 0.14212532395118227, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.14)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: left, reward: 1.84931281162
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 24, 't': 11, 'action': 'left', 'reward': 1.849312811621703, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.85)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: -0.132309158276
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 23, 't': 12, 'action': 'right', 'reward': -0.13230915827643452, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded -0.13)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 0.852064253078
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 22, 't': 13, 'action': None, 'reward': 0.852064253078334, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 0.85)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: 2.26975047656
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 21, 't': 14, 'action': 'right', 'reward': 2.269750476556618, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.27)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 1.64989762197
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 15, 'action': 'right', 'reward': 1.649897621971391, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.65)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 2.10625324176
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 16, 'action': None, 'reward': 2.106253241756258, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.11)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 1.0510105768
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 18, 't': 17, 'action': 'forward', 'reward': 1.0510105768005864, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.05)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: left, reward: -0.207532011509
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 17, 't': 18, 'action': 'left', 'reward': -0.20753201150924294, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded -0.21)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: 2.69628774192
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 16, 't': 19, 'action': None, 'reward': 2.6962877419209215, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.70)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: 1.4549226937
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 20, 'action': None, 'reward': 1.4549226937039939, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.45)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 0.711693836462
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 14, 't': 21, 'action': 'right', 'reward': 0.7116938364615264, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 0.71)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: -39.2627896757
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', 'right'), 'deadline': 13, 't': 22, 'action': 'forward', 'reward': -39.26278967567875, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', 'right')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.26)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 2.07572033416
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 23, 'action': None, 'reward': 2.0757203341593238, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.08)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 1.82292304957
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 11, 't': 24, 'action': 'right', 'reward': 1.822923049566095, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.82)
29% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: forward, reward: 0.843074319971
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'left'), 'deadline': 10, 't': 25, 'action': 'forward', 'reward': 0.8430743199708723, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'left')
Agent drove forward instead of right. (rewarded 0.84)
26% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: 1.97965253332
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 9, 't': 26, 'action': None, 'reward': 1.979652533319685, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.98)
23% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: 1.72242892222
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 8, 't': 27, 'action': 'right', 'reward': 1.722428922221138, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 1.72)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 423
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (7, 2), deadline = 30
Simulating trial. . .
epsilon = 0.0146; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.29745194467
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.2974519446734902, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.30)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.48209766638
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.4820976663813392, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.48)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 2.11925246829
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.119252468287386, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.12)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 2.27713370696
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.277133706960248, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.28)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.24273137951
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.2427313795089614, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.24)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 2.33376967647
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 2.333769676466586, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.33)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 2.06988798466
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 2.0698879846588616, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.07)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: 1.14833000801
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 1.1483300080057866, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.15)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 2.44472328722
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.4447232872215614, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.44)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 2.62364450102
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 2.623644501018523, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.62)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 1.49422531233
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.4942253123271054, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 1.49)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 2.16179702097
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.1617970209731934, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.16)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.34701018531
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'right'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 1.3470101853097713, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 1.35)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 0.0146048431953
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 0.014604843195322292, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.01)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 1.29421612461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.2942161246111128, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.29)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 2.52320145278
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 2.5232014527752473, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.52)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: forward, reward: 0.554492723211
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': 0.5544927232109125, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.55)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 2.31255889401
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 2.3125588940137165, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.31)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 0.310139035909
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 0.31013903590890557, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 0.31)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: 2.6247867076
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 19, 'action': 'left', 'reward': 2.624786707603663, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.62)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 1.41156270088
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 10, 't': 20, 'action': 'forward', 'reward': 1.4115627008779648, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove forward instead of left. (rewarded 1.41)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: left, reward: 1.64627466441
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'left', 'reward': 1.6462746644058048, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.65)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 1.67759671736
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', None), 'deadline': 8, 't': 22, 'action': None, 'reward': 1.6775967173601547, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 1.68)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 1.04787877439
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 7, 't': 23, 'action': None, 'reward': 1.0478787743883025, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.05)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 1.5010290124
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 6, 't': 24, 'action': None, 'reward': 1.5010290123960535, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.50)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 1.17237174871
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 1.1723717487131917, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove right instead of left. (rewarded 1.17)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: 1.70043899643
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 4, 't': 26, 'action': 'left', 'reward': 1.7004389964330704, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.70)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: 1.14418672484
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 3, 't': 27, 'action': None, 'reward': 1.1441867248369766, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.14)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: left, reward: 1.95779314709
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 28, 'action': 'left', 'reward': 1.9577931470857104, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.96)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 0.593217383606
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 1, 't': 29, 'action': None, 'reward': 0.5932173836055161, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.59)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 424
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (7, 7), deadline = 30
Simulating trial. . .
epsilon = 0.0144; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 2.45511079297
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'right'), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.4551107929675844, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 2.46)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 2.47804790162
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.478047901618156, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.48)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 1.80825307697
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.8082530769671268, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.81)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 1.43816371796
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.4381637179551259, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.44)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 1.76238305218
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.7623830521781412, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.76)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 2.86479992358
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.8647999235833006, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.86)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: 1.61061834197
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 1.6106183419681976, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.61)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.41027230356
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 23, 't': 7, 'action': None, 'reward': 2.4102723035645015, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.41)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.85400709031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.8540070903117567, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.85)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 2.35300838015
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 2.3530083801548223, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.35)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 1.44251501509
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.4425150150880888, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.44)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: left, reward: 2.29803831264
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 2.29803831264132, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.30)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 1.00728078699
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.0072807869908915, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.01)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 1.30561904066
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.3056190406557553, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.31)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 2.29647118866
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 2.296471188662211, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 2.30)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: forward, reward: 2.22970158016
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': 2.2297015801644378, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.23)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 0.848570336758
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 16, 'action': None, 'reward': 0.8485703367582893, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.85)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: forward, reward: 1.20693857192
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': 1.206938571922879, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.21)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 425
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (2, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0143; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0143; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0143; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 2.28075330464
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.280753304642133, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.28)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 1.96922425639
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.9692242563949887, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.97)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 2.9262369464
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.9262369463952256, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.93)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: 2.44459173615
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.4445917361481957, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.44)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: left, reward: 2.61934497058
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 2.619344970578635, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.62)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 2.23637848022
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.236378480221453, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.24)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: 1.91076142875
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.9107614287494727, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.91)
65% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 426
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (5, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0141; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 1.87005043264
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.870050432637115, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.87)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 1.31805561951
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.3180556195083766, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.32)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.07838551855
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.0783855185504478, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.08)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 2.9450276383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.945027638300675, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.95)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.65565906922
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.655659069215334, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.66)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 2.85981659392
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.8598165939229556, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.86)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.02307900564
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.0230790056355954, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.02)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.26152661352
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.2615266135233985, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.26)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.14523549501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.1452354950130392, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.15)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 1.62947733812
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.6294773381167111, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.63)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 0.538298227357
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.5382982273568128, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 0.54)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 2.6022800143
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.6022800142983438, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.60)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 2.59046596151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.5904659615112156, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.59)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: left, reward: 1.02211147952
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 1.0221114795249318, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.02)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 0.815988740031
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 0.8159887400305593, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove right instead of left. (rewarded 0.82)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: 2.58923773328
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 2.589237733280863, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.59)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 0.701927230123
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 9, 't': 16, 'action': None, 'reward': 0.7019272301231962, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 0.70)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.53633681317
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 8, 't': 17, 'action': None, 'reward': 1.536336813171506, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.54)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: right, reward: 2.26768298741
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 2.267682987412038, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 2.27)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 1.19471886512
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 1.1947188651211236, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.19)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 0.694974913536
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.6949749135362957, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 0.69)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: -0.175945671384
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': -0.1759456713842399, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded -0.18)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: left, reward: 1.39215758853
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 3, 't': 22, 'action': 'left', 'reward': 1.3921575885307425, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.39)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: left, reward: 0.583246408243
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 2, 't': 23, 'action': 'left', 'reward': 0.5832464082434612, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.58)
4% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 427
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (5, 2), deadline = 35
Simulating trial. . .
epsilon = 0.0140; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: left, reward: 1.85034568154
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 35, 't': 0, 'action': 'left', 'reward': 1.8503456815360428, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.85)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.78462261223
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 34, 't': 1, 'action': None, 'reward': 1.784622612228405, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.78)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 2.68239280824
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 33, 't': 2, 'action': None, 'reward': 2.6823928082386486, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.68)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.65925218399
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 32, 't': 3, 'action': None, 'reward': 1.6592521839891199, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.66)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 2.33672854445
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 31, 't': 4, 'action': None, 'reward': 2.336728544450618, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.34)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.84633628723
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 30, 't': 5, 'action': None, 'reward': 1.8463362872337767, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.85)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: left, reward: 1.42333931389
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 29, 't': 6, 'action': 'left', 'reward': 1.4233393138931054, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.42)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: 1.80755922928
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'forward'), 'deadline': 28, 't': 7, 'action': 'forward', 'reward': 1.807559229278164, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'forward')
Agent drove forward instead of right. (rewarded 1.81)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 2.81648289349
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 27, 't': 8, 'action': None, 'reward': 2.8164828934941193, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.82)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 2.15963969216
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 26, 't': 9, 'action': None, 'reward': 2.159639692160779, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.16)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: right, reward: 2.32151691264
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 25, 't': 10, 'action': 'right', 'reward': 2.3215169126371897, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.32)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 1.23353926417
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 24, 't': 11, 'action': 'forward', 'reward': 1.2335392641699647, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.23)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: 0.485056694897
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', 'left'), 'deadline': 23, 't': 12, 'action': 'left', 'reward': 0.48505669489682235, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', 'left')
Agent drove left instead of forward. (rewarded 0.49)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: forward, reward: 0.605536480158
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 22, 't': 13, 'action': 'forward', 'reward': 0.6055364801575714, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 0.61)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: right, reward: 1.79304589611
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 21, 't': 14, 'action': 'right', 'reward': 1.793045896107535, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.79)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: right, reward: 1.10106642088
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 20, 't': 15, 'action': 'right', 'reward': 1.1010664208814736, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.10)
54% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 428
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (5, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0138; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0138; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.77484432522
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.7748443252172572, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.77)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 2.89569449107
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.8956944910691416, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.90)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.26659679002
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.2665967900153001, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.27)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 2.14804451104
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 2.1480445110410447, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.15)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 1.8080762587
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.8080762586960604, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 1.81)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 1.11557817196
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.1155781719601754, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.12)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: left, reward: 2.28518353205
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.2851835320451883, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.29)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: forward, reward: 1.1757919932
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.175791993199162, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.18)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: left, reward: 2.0382320813
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 2.0382320813022563, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.04)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 429
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (2, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0137; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0137; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: left, reward: 1.50939861799
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.5093986179896892, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.51)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: left, reward: 0.856374945962
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 0.8563749459619875, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.86)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 1.78599001509
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.7859900150908916, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.79)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 1.24644050666
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.246440506659796, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.25)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 2.72522471325
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.7252247132544163, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.73)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: 2.29241691055
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 2.292416910551542, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.29)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 0.928390347312
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 0.9283903473116575, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.93)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.08698942712
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.086989427119169, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.09)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: -9.50864676415
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -9.508646764149207, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.51)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: forward, reward: 1.95705126079
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.9570512607929231, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.96)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 430
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (6, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0136; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0136; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0136; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: -19.0916906939
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'left'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': -19.091690693896297, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.09)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 1.19626641186
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.1962664118596995, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.20)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 0.958782161968
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 0.958782161967801, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 0.96)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: left, reward: 1.00167277008
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 1.001672770082582, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.00)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 0.977637896715
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 0.9776378967149166, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.98)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 2.70696350205
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 2.706963502045438, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.71)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: forward, reward: 1.46142502265
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.4614250226549514, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.46)
72% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 431
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (3, 7), deadline = 25
Simulating trial. . .
epsilon = 0.0134; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0134; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0134; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: forward, reward: 1.51904128347
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.519041283472703, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.52)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 2.80950490708
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.809504907081002, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.81)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 1.27927585124
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.2792758512398643, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.28)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 0.965270764658
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 0.9652707646577314, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.97)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 1.12270324264
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.1227032426356773, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.12)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: 1.09302049334
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.0930204933380103, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent followed the waypoint forward. (rewarded 1.09)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 2.36751356272
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.3675135627243864, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.37)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 2.32202172831
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.3220217283085187, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.32)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 2.15832840093
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'forward', 'left'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 2.1583284009268784, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'forward', 'left')
Agent followed the waypoint right. (rewarded 2.16)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: forward, reward: 2.7873176137
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.7873176137046345, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.79)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 432
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (8, 7), deadline = 25
Simulating trial. . .
epsilon = 0.0133; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: forward, reward: 1.38407848584
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', 'left'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.3840784858350326, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', 'left')
Agent drove forward instead of right. (rewarded 1.38)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: right, reward: 2.00272821919
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.002728219189657, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.00)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: 1.88003026468
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 1.8800302646811966, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.88)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 2.49248245206
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.4924824520593667, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.49)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.9823281785
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.982328178495633, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.98)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.49225106715
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.4922510671450289, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.49)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.4236110264
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.4236110264013158, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.42)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: left, reward: 0.663618270398
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 0.6636182703984399, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', None)
Agent drove left instead of forward. (rewarded 0.66)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 2.12130732578
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 2.121307325778938, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.12)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 2.24834095933
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 2.24834095932547, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.25)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 2.53316387473
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.5331638747262284, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.53)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: forward, reward: 1.87478016328
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.8747801632790444, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.87)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: forward, reward: 2.01507774017
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 2.015077740169114, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.02)
48% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 433
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (3, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0132; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0132; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0132; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0132; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 0.792942358248
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 0.7929423582475603, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 0.79)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: left, reward: 0.97516047118
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 0.9751604711795889, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', None)
Agent drove left instead of forward. (rewarded 0.98)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 1.33354683205
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.3335468320517845, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.33)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 1.76155876993
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.7615587699307909, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.76)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 2.37578193057
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.3757819305742176, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.38)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 2.79425944473
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.794259444734006, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.79)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 2.85616636315
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.8561663631544727, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.86)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.55144424144
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.5514442414392613, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.55)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: -0.083239422939
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': -0.08323942293902886, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded -0.08)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 2.58960916448
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.589609164479807, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 2.59)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 2.13855581611
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.1385558161056535, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.14)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 0.686844229419
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 0.686844229418813, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove right instead of left. (rewarded 0.69)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 1.0093926211
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.0093926210982256, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.01)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 0.755795711034
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 0.7557957110343225, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.76)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 2.51921259158
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 2.5192125915825088, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.52)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 1.28515212379
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 1.2851521237928696, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.29)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: 2.58553713221
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 2.585537132212094, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.59)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.15714467682
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 8, 't': 17, 'action': None, 'reward': 1.157144676823106, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.16)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.37188796005
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.371887960048891, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.37)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 2.23218499793
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 6, 't': 19, 'action': None, 'reward': 2.2321849979344615, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.23)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 0.983451664472
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.9834516644722473, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 0.98)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 0.041592882936
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'right', 'left'), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 0.04159288293604291, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'right', 'left')
Agent drove right instead of forward. (rewarded 0.04)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: 1.48864580864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 3, 't': 22, 'action': 'left', 'reward': 1.4886458086359402, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.49)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.78700080989
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.7870008098862553, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.79)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.42772100519
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.4277210051942082, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.43)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 434
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (3, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0130; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0130; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: left, reward: 1.18695980698
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.1869598069804308, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent drove left instead of forward. (rewarded 1.19)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 1.10944481531
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.109444815308851, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.11)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.51045021042
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.5104502104185205, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.51)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 0.197269587915
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 0.19726958791530802, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove right instead of forward. (rewarded 0.20)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: left, reward: 2.56941388624
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 2.5694138862385647, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 2.57)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 1.68509592251
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.6850959225091628, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.69)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 1.49766243219
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.4976624321924028, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.50)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: left, reward: 2.37332008434
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 2.3733200843430975, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.37)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 435
\-------------------------
Environment.reset(): Trial set up with start = (7, 3), destination = (3, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0129; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: 2.79697334622
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 2.796973346221381, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.80)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 2.61116502762
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.611165027617946, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.61)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 1.90601652508
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.9060165250842025, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.91)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: left, reward: 0.259457255077
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 0.2594572550770766, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.26)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 1.29984707141
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.29984707140845, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.30)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 1.72506775722
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.7250677572151127, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.73)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 1.70572357642
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.7057235764232934, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.71)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: forward, reward: 1.58621555903
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.5862155590285056, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.59)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 2.03795733711
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.037957337106821, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.04)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: 1.10075704702
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'right'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.100757047020465, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 1.10)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 2.80630395335
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.8063039533459566, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.81)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: 2.13928884645
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 2.1392888464469997, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 2.14)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 0.822550725122
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 0.8225507251216091, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.82)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 1.3083130943
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.3083130942992847, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.31)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 1.82692310763
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.8269231076342876, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.83)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: 1.36450207105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.36450207104926, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.36)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: left, reward: 0.757995349833
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 0.7579953498331167, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 0.76)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 0.678234332719
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 0.6782343327192317, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 0.68)
28% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 436
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (1, 2), deadline = 30
Simulating trial. . .
epsilon = 0.0128; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.18966295459
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', 'right'), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.189662954591901, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', 'right')
Agent properly idled at a red light. (rewarded 2.19)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.25735173492
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.2573517349249217, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.26)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 1.10025466198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.1002546619765523, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.10)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.34064523666
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.3406452366577275, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.34)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.68223774637
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.682237746367731, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.68)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: left, reward: 2.30949139146
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 2.309491391460101, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 2.31)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 1.4928740113
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.4928740112973007, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.49)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: left, reward: 1.96379347949
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 1.963793479490924, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.96)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 0.925029396055
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 8, 'action': None, 'reward': 0.9250293960546068, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.93)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 0.502701786952
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 0.5027017869518307, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.50)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: 0.926785525136
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 0.9267855251355728, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 0.93)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 2.8355432301
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.8355432301015364, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.84)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 2.25749108125
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.2574910812516764, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.26)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: -0.0910653350968
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 17, 't': 13, 'action': 'right', 'reward': -0.09106533509683623, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded -0.09)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: forward, reward: 0.27007729389
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'forward'), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': 0.2700772938904379, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'forward')
Agent drove forward instead of left. (rewarded 0.27)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 1.44937250853
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.449372508526764, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.45)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: left, reward: 1.57291751012
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 1.5729175101240647, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.57)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.39597014879
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.395970148790061, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.40)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.66454768426
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.6645476842604177, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.66)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: left, reward: -0.198163195816
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 11, 't': 19, 'action': 'left', 'reward': -0.19816319581565411, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded -0.20)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 1.82660645041
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'right'), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 1.8266064504138835, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 1.83)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 0.608810702037
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 21, 'action': None, 'reward': 0.608810702037371, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.61)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: 0.930438109092
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 8, 't': 22, 'action': 'left', 'reward': 0.9304381090922538, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 0.93)
23% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 437
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (8, 7), deadline = 25
Simulating trial. . .
epsilon = 0.0127; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 1.96728848513
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.9672884851347194, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.97)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 1.75635841726
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.7563584172579167, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.76)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 2.83873347022
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.8387334702177482, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.84)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 1.01016689402
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.0101668940242183, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.01)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 2.1211942606
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.121194260603325, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.12)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 1.33941014255
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.3394101425458034, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.34)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: left, reward: 2.34105324462
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 2.341053244621307, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.34)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.27531049198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.2753104919787726, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 2.28)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.34246163825
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.3424616382482069, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.34)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.30029875303
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.3002987530264816, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.30)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 1.55807411374
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.558074113737081, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.56)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: left, reward: 1.33752212824
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.337522128244523, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.34)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 1.43433369354
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.434333693542942, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.43)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 0.757920160373
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 12, 't': 13, 'action': None, 'reward': 0.7579201603727503, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.76)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: -0.152984059497
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': -0.15298405949742888, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded -0.15)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 0.402428337088
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 0.40242833708799886, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.40)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 0.805791919178
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 0.8057919191784502, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 0.81)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 2.51116471413
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 2.5111647141329625, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.51)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: left, reward: 0.787134589319
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 18, 'action': 'left', 'reward': 0.7871345893192694, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.79)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 1.14863377983
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 1.1486337798255202, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.15)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 438
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (8, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0125; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 2.17536422383
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.1753642238290762, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.18)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: -40.1501730461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': 'right'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', 'right'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -40.150173046087176, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.15)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: right, reward: 0.381078894041
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 0.3810788940405283, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', None)
Agent drove right instead of forward. (rewarded 0.38)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: left, reward: 2.36652285039
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 2.3665228503905933, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.37)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.59915380019
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.5991538001946008, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.60)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 1.9271190922
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.9271190922030446, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.93)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: left, reward: 1.74110094541
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.7411009454058421, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.74)
72% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 439
\-------------------------
Environment.reset(): Trial set up with start = (2, 7), destination = (8, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0124; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0124; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0124; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0124; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 2.44127891102
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.4412789110153814, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.44)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: left, reward: 0.515596404138
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 0.5155964041376376, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent drove left instead of right. (rewarded 0.52)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.50717353485
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.5071735348543684, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.51)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 1.81799241674
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.8179924167432957, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.82)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 2.6559552976
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.6559552975999674, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.66)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.28440592193
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.2844059219306168, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.28)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 2.24339552827
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.24339552826888, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.24)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 0.980600093989
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 0.9806000939891804, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.98)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.4955470287
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.4955470286954706, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.50)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 2.70774658054
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 2.7077465805424303, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.71)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 0.815527503639
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 0.815527503639085, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 0.82)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.44500381858
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.445003818582418, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.45)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.15044600032
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.1504460003181265, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.15)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 2.01557034266
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 2.015570342656958, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.02)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 440
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (2, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0123; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 2.33572347356
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.335723473558418, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.34)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 1.88516123038
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.885161230383084, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.89)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 2.87276819015
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 2.872768190146613, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.87)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 0.0495837271593
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 0.04958372715934234, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 0.05)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: 1.11814146192
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.1181414619154153, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.12)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.68975008277
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.6897500827697907, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.69)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: 1.08447186573
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.084471865725113, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.08)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 1.95160490044
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.9516049004380702, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.95)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: 0.461996774264
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 0.4619967742640766, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.46)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 2.29169105061
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.2916910506053743, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.29)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 0.978654087632
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 0.9786540876321594, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.98)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: left, reward: 1.27133290981
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 1.271332909810852, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 1.27)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 1.58426121153
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.584261211533516, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 1.58)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 0.819839369505
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.8198393695049762, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 0.82)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: -19.479583538
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', 'right'), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -19.479583537953978, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.48)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 2.34231857663
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 2.3423185766284096, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 2.34)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.02104140042
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 4, 't': 16, 'action': None, 'reward': 2.021041400423715, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.02)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.9862758545
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.9862758544976007, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.99)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: -0.130715618157
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': -0.13071561815699062, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded -0.13)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 0.317488030193
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.31748803019341354, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 0.32)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 441
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (1, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0122; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.46051083092
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.4605108309246684, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.46)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.14439950056
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.1443995005567613, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.14)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.39955230319
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.399552303185753, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.40)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.29991975561
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.2999197556077182, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.30)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.32943562994
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.329435629944069, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.33)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.39804399858
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.398043998577905, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.40)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: left, reward: 2.07702942901
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 2.0770294290127476, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.08)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 1.07789571495
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.0778957149458543, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove right instead of left. (rewarded 1.08)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: forward, reward: 0.251130121818
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 0.2511301218183307, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 0.25)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: forward, reward: 0.494994002369
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 0.49499400236933755, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.49)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: 2.68183280892
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 2.6818328089205137, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.68)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: 0.722390050365
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 0.7223900503652371, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.72)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 1.63792602214
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.6379260221421061, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.64)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 2.2805136436
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 12, 't': 13, 'action': None, 'reward': 2.280513643600575, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.28)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 1.60452316716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.6045231671560276, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.60)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 2.08006963204
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.080069632035544, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.08)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: 2.25241261146
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 2.2524126114594454, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 2.25)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.72500350419
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 17, 'action': None, 'reward': 1.7250035041897755, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.73)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.67279229707
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.6727922970715101, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.67)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.88775259338
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 6, 't': 19, 'action': None, 'reward': 1.8877525933766777, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.89)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: right, reward: 0.80694418281
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', 'left'), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 0.8069441828098973, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', 'left')
Agent drove right instead of forward. (rewarded 0.81)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: 0.509384155041
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 0.5093841550409764, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 0.51)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 0.639471018742
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 3, 't': 22, 'action': None, 'reward': 0.6394710187419321, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.64)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 1.70798336013
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.7079833601274677, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.71)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 1.61166339008
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.6116633900799084, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.61)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 442
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (7, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0120; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0120; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 2.83029162219
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.830291622194596, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.83)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 2.84987634562
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.8498763456244873, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent properly idled at a red light. (rewarded 2.85)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 2.75247269665
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.752472696652359, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.75)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: 1.57261365285
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.5726136528489691, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.57)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: forward, reward: 0.672962065377
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 0.6729620653770473, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded 0.67)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 0.318434331006
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.3184343310061828, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 0.32)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: 1.50770424246
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.507704242459601, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.51)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: left, reward: 2.85073322472
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 2.8507332247244013, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.85)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 1.30499181798
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.3049918179793372, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.30)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: 2.12206975857
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 2.1220697585703765, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.12)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.58435099533
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.5843509953336858, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.58)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 2.28628574394
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.286285743938782, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.29)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 0.829239336992
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.8292393369916902, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 0.83)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 1.58465208361
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.5846520836110598, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent followed the waypoint forward. (rewarded 1.58)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 1.69784478539
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.6978447853889744, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.70)
25% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 443
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (2, 5), deadline = 30
Simulating trial. . .
epsilon = 0.0119; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 1.15357930813
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.153579308130365, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.15)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 2.08326694372
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.083266943716087, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.08)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 1.53186652366
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.5318665236566509, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.53)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 2.33552617216
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.335526172155291, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.34)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: left, reward: 1.41657903069
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 1.4165790306905388, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.42)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: left, reward: 2.01749253433
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 2.0174925343345382, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 2.02)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 0.953010458862
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'forward'), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 0.953010458862278, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.95)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.83972853755
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.839728537547093, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.84)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.02581720155
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.025817201553547, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.03)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: right, reward: 1.29781661938
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 1.2978166193753995, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.30)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: -0.0508184532594
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': -0.05081845325940981, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded -0.05)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 1.52270049408
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.5227004940823745, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.52)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: 0.907428065366
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': 0.9074280653662594, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 0.91)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 0.174983272578
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 0.17498327257773227, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.17)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: right, reward: 0.85389221382
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 0.8538922138196512, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.85)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: right, reward: 1.01562992684
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.0156299268422417, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.02)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: 1.44287549081
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 1.4428754908123285, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.44)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 0.871607342539
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 0.8716073425387836, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.87)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 1.87110870131
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.8711087013120256, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.87)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: 2.2551451456
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 11, 't': 19, 'action': 'left', 'reward': 2.2551451456030898, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.26)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: left, reward: 2.25931356362
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 20, 'action': 'left', 'reward': 2.259313563622455, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.26)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 444
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (1, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0118; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 2.63904634799
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.6390463479881205, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.64)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: right, reward: 1.77837472208
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.7783747220814605, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.78)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: right, reward: 1.40616131249
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'right'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.4061613124911894, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'right')
Agent drove right instead of forward. (rewarded 1.41)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: forward, reward: 1.54079949829
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'forward'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.5407994982923112, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.54)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: left, reward: 1.4341189702
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.4341189702048596, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.43)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.79113579632
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.7911357963214183, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.79)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 1.80268058821
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.8026805882110988, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.80)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 1.99914478556
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.9991447855632543, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.00)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 1.16791482254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.1679148225356515, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.17)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 1.23527186181
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.2352718618091916, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.24)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: 2.42883789325
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 2.428837893248363, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.43)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 1.05165552038
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.0516555203797808, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.05)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 1.47642890941
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.4764289094138512, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.48)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 0.139575750224
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'right'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.13957575022366842, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'right')
Agent drove right instead of forward. (rewarded 0.14)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.43323931896
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'right'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.4332393189649553, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.43)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 2.23228773201
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.2322877320148793, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.23)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.27682383626
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.276823836264419, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.28)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 2.29149114271
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 2.2914911427080837, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.29)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 0.329199919953
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 0.32919991995277953, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 0.33)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: -0.270074797495
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -0.27007479749539975, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded -0.27)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 445
\-------------------------
Environment.reset(): Trial set up with start = (7, 3), destination = (4, 7), deadline = 25
Simulating trial. . .
epsilon = 0.0117; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 2.7514128889
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.7514128888995666, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.75)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 2.03554975154
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.0355497515376793, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.04)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 1.21461099252
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.2146109925162494, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.21)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 1.90990259127
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.9099025912682945, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.91)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 2.33043248349
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.3304324834886696, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.33)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: 0.726097035356
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 0.7260970353563084, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 0.73)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 1.52590881856
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.5259088185571328, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.53)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 1.66605492512
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.6660549251186412, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.67)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 0.904851763638
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 0.9048517636377329, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 0.90)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 0.11064401663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.11064401663028356, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.11)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: left, reward: 2.51586750124
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 2.5158675012399656, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 2.52)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: 1.25021563363
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.2502156336333046, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.25)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: forward, reward: 0.511701778725
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 0.5117017787247287, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 0.51)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: forward, reward: 1.62076960838
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 1.6207696083837129, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.62)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: left, reward: 2.37983895412
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 2.37983895412476, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 2.38)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: 2.29327678864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 2.2932767886399055, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.29)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 1.1187602793
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.1187602792971942, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.12)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: left, reward: 1.06590151112
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 1.0659015111190193, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 1.07)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: right, reward: 0.64205593201
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 0.6420559320097021, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.64)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 0.959703158306
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 0.9597031583055431, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 0.96)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 2.01085857292
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 2.0108585729190693, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.01)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: forward, reward: 0.49288062198
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 0.49288062198019045, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.49)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 2.18260001644
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 3, 't': 22, 'action': None, 'reward': 2.1826000164416977, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.18)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 1.38253617309
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.3825361730925951, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.38)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 0.210644950008
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.21064495000825278, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 0.21)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 446
\-------------------------
Environment.reset(): Trial set up with start = (8, 7), destination = (6, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0116; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 2.30272111952
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.3027211195161055, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.30)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: right, reward: 2.74758522022
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.747585220220259, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.75)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.20779565446
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.2077956544583497, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.21)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 1.5650152715
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.5650152714963785, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.57)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.20259457043
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.2025945704298038, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.20)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 1.43855092482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.4385509248175643, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.44)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 1.11145673363
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'right', 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.1114567336264043, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'right', 'forward')
Agent drove right instead of forward. (rewarded 1.11)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: left, reward: 1.94895803166
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.9489580316573096, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'left')
Agent followed the waypoint left. (rewarded 1.95)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 0.648025900807
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.6480259008067338, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent drove right instead of left. (rewarded 0.65)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: forward, reward: 2.73642887127
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 2.736428871270972, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.74)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: forward, reward: 2.32524732905
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 2.3252473290536626, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.33)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 447
\-------------------------
Environment.reset(): Trial set up with start = (4, 7), destination = (8, 4), deadline = 35
Simulating trial. . .
epsilon = 0.0114; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 2.68973053663
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'right'), 'deadline': 35, 't': 0, 'action': 'right', 'reward': 2.6897305366299276, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 2.69)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: 2.98296886901
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 34, 't': 1, 'action': 'right', 'reward': 2.9829688690092615, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.98)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 1.69996990884
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 33, 't': 2, 'action': None, 'reward': 1.6999699088389988, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.70)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 2.07670104855
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 32, 't': 3, 'action': 'forward', 'reward': 2.076701048547863, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.08)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: left, reward: 1.34626262253
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 31, 't': 4, 'action': 'left', 'reward': 1.3462626225276961, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.35)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 1.34453600951
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 30, 't': 5, 'action': None, 'reward': 1.344536009510997, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.34)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 1.35109004373
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 29, 't': 6, 'action': 'right', 'reward': 1.3510900437309694, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.35)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.469630722
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 28, 't': 7, 'action': None, 'reward': 1.4696307220007234, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.47)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 0.945561266999
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 27, 't': 8, 'action': None, 'reward': 0.9455612669992486, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 0.95)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.80788226465
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'right'), 'deadline': 26, 't': 9, 'action': None, 'reward': 2.80788226465164, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 2.81)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 1.83677425513
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 25, 't': 10, 'action': 'forward', 'reward': 1.8367742551268502, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.84)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 2.02473856885
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 24, 't': 11, 'action': None, 'reward': 2.0247385688548913, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.02)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 2.64383128112
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 23, 't': 12, 'action': None, 'reward': 2.643831281123295, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.64)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 1.0288800029
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 22, 't': 13, 'action': 'forward', 'reward': 1.0288800029022687, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.03)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: left, reward: 2.29635097141
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 21, 't': 14, 'action': 'left', 'reward': 2.2963509714144212, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.30)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 1.27701551479
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 15, 'action': None, 'reward': 1.2770155147863445, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.28)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: forward, reward: 0.188444006855
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 19, 't': 16, 'action': 'forward', 'reward': 0.18844400685482976, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.19)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: 1.21622245862
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 18, 't': 17, 'action': 'left', 'reward': 1.2162224586173402, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.22)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.0032642057
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 18, 'action': None, 'reward': 1.0032642056994645, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.00)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.35982693033
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 16, 't': 19, 'action': None, 'reward': 2.3598269303345774, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.36)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.48137253335
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 20, 'action': None, 'reward': 2.4813725333516965, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.48)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 1.36878157097
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 14, 't': 21, 'action': 'right', 'reward': 1.3687815709719149, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 1.37)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 2.43870925122
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 13, 't': 22, 'action': None, 'reward': 2.4387092512165354, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.44)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: -0.205922709736
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'forward'), 'deadline': 12, 't': 23, 'action': 'forward', 'reward': -0.20592270973610516, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'forward')
Agent drove forward instead of right. (rewarded -0.21)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: right, reward: -0.31041694624
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 11, 't': 24, 'action': 'right', 'reward': -0.3104169462402626, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded -0.31)
29% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.20944460121
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 25, 'action': None, 'reward': 2.209444601211441, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.21)
26% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: left, reward: 1.54685149367
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 9, 't': 26, 'action': 'left', 'reward': 1.5468514936653202, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.55)
23% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.61018412068
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 8, 't': 27, 'action': None, 'reward': 1.6101841206801617, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.61)
20% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 2.20543648047
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 7, 't': 28, 'action': None, 'reward': 2.205436480474141, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.21)
17% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: 1.37143329157
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 6, 't': 29, 'action': 'left', 'reward': 1.3714332915665601, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.37)
14% of time remaining to reach destination.
/-------------------
| Step 30 Results
\-------------------
Environment.step(): t = 30
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: -0.435993400389
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 5, 't': 30, 'action': 'forward', 'reward': -0.43599340038931933, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded -0.44)
11% of time remaining to reach destination.
/-------------------
| Step 31 Results
\-------------------
Environment.step(): t = 31
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: left, reward: -0.00368408752487
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 4, 't': 31, 'action': 'left', 'reward': -0.0036840875248730365, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove left instead of right. (rewarded -0.00)
9% of time remaining to reach destination.
/-------------------
| Step 32 Results
\-------------------
Environment.step(): t = 32
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 0.647505252273
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 3, 't': 32, 'action': None, 'reward': 0.6475052522734404, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.65)
6% of time remaining to reach destination.
/-------------------
| Step 33 Results
\-------------------
Environment.step(): t = 33
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: -0.529461864109
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 2, 't': 33, 'action': 'right', 'reward': -0.529461864108915, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove right instead of left. (rewarded -0.53)
3% of time remaining to reach destination.
/-------------------
| Step 34 Results
\-------------------
Environment.step(): t = 34
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 1.11368259373
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'forward'), 'deadline': 1, 't': 34, 'action': None, 'reward': 1.1136825937342907, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'left', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.11)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 448
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (1, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0113; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: None, reward: 1.28835435747
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.2883543574659224, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.29)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: None, reward: 2.66548163022
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.665481630218493, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.67)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: None, reward: 1.25985063555
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.2598506355488501, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.26)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: forward, reward: 1.37046063943
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.3704606394298953, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.37)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 2.30830416602
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.3083041660208794, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.31)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 1.7478749904
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.7478749903997266, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.75)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: 1.64644833607
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.6464483360736757, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.65)
65% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 449
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (8, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0112; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 2.99365975203
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'left'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.993659752031438, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'left')
Agent followed the waypoint forward. (rewarded 2.99)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 1.09379187309
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.0937918730895824, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 1.09)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: left, reward: 2.6645807001
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 2.664580700098062, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.66)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 2.28492859982
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.2849285998190094, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.28)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 1.69471624279
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.6947162427920401, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 1.69)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 2.14314188313
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.1431418831254936, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.14)
70% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 450
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (3, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0111; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: left, reward: 1.70927704655
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.709277046549793, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 1.71)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 2.23487842682
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.2348784268224144, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.23)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 2.6242968151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.624296815098007, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.62)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: left, reward: 2.86076817841
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 2.8607681784139287, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.86)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 2.92086793966
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.9208679396560226, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.92)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 1.5626926011
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.5626926011042201, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 1.56)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 1.89344711501
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.8934471150090397, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 1.89)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 2.26493062029
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.2649306202896593, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.26)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 0.878678377768
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 0.8786783777679272, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.88)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 1.04654187498
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'right'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.046541874978663, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'right')
Agent drove forward instead of right. (rewarded 1.05)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.34730182269
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.3473018226869022, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.35)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: left, reward: 1.33132253842
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 1.3313225384199854, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent drove left instead of right. (rewarded 1.33)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: forward, reward: 0.296914723979
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'left'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 0.2969147239791241, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'left')
Agent drove forward instead of left. (rewarded 0.30)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: 0.779197409977
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.7791974099773669, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.78)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.24518706148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.2451870614764562, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.25)
25% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 451
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (6, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0110; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: -19.1435879008
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', 'left'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': -19.143587900841414, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.14)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 2.0463243579
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.046324357896965, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.05)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 2.72073821961
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.7207382196107837, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.72)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 1.48776816881
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.4877681688070101, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.49)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 1.73160535106
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.7316053510602187, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 1.73)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: left, reward: 1.84055724585
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.8405572458502883, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.84)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 2.1345887315
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.1345887314977308, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.13)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: forward, reward: 1.56534293782
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.56534293782398, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.57)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: left, reward: 1.08413665583
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 1.0841366558268413, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.08)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: right, reward: 1.61260554547
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.6126055454664607, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.61)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 2.43564150138
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 2.4356415013806227, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.44)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 452
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (7, 4), deadline = 30
Simulating trial. . .
epsilon = 0.0109; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: left, reward: 2.60102325181
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 2.6010232518090124, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 2.60)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 2.1557857695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.1557857694999942, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.16)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 2.77202026421
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.772020264207385, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.77)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 2.36224781661
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.3622478166141, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.36)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 2.90735512121
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.907355121212188, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.91)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: 1.76433843621
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.764338436208251, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.76)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.85427314526
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.8542731452634527, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.85)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: 2.11996720989
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 2.119967209886032, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent followed the waypoint forward. (rewarded 2.12)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.27571242817
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.275712428171074, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.28)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 2.46836430172
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 2.46836430171766, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.47)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 1.06396569751
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.0639656975135938, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.06)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 1.12686746227
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.1268674622746115, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.13)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 2.06725942774
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.0672594277421803, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.07)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 2.10372216465
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 17, 't': 13, 'action': None, 'reward': 2.103722164650652, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.10)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 1.89819701397
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.8981970139714084, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.90)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 0.989128570901
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 15, 'action': None, 'reward': 0.9891285709013669, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.99)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: left, reward: 0.471432646557
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 0.4714326465568308, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.47)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 0.0206666184257
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 13, 't': 17, 'action': None, 'reward': 0.020666618425738248, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.02)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: right, reward: 2.08888102123
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 2.0888810212333118, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.09)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: right, reward: 1.75538008675
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 1.7553800867531961, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.76)
33% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 453
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (5, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0108; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 2.19890722124
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.198907221238091, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 2.20)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 1.04583233004
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.0458323300412953, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.05)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 1.68676288156
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.6867628815574875, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.69)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 1.74458288725
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.7445828872478875, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.74)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 2.2072109744
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.2072109743968027, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.21)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: 2.31602244427
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.316022444268119, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.32)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: left, reward: 2.05846749498
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 2.058467494981672, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 2.06)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: None, reward: 2.56661539128
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.5666153912841834, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.57)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: None, reward: 2.75901640528
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.759016405282133, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.76)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: None, reward: 2.8079926691
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.8079926691006207, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.81)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: forward, reward: 2.1033218781
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 2.1033218780961356, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.10)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: forward, reward: 0.971213993889
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 0.9712139938894797, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.97)
52% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 454
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (8, 3), deadline = 30
Simulating trial. . .
epsilon = 0.0107; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0107; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.5010607726
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.5010607725980174, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.50)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.33133320577
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.331333205766757, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.33)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.81832956984
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.8183295698379895, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.82)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.5694297937
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.5694297937004218, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.57)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.45316187658
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.453161876575133, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.45)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.10372458883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.1037245888298406, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.10)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: 1.90833017207
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 1.9083301720689039, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove forward instead of left. (rewarded 1.91)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: 2.83608409232
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 2.8360840923150317, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 2.84)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.64447554362
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.644475543615906, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.64)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 1.6132615909
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'forward'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.6132615908977697, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.61)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 1.06788482163
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.0678848216305912, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.07)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 2.39459230765
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 2.3945923076541584, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 2.39)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.9075570034
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.9075570033952463, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.91)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.48780738435
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 13, 'action': None, 'reward': 1.4878073843489323, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.49)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 2.40017331802
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 2.4001733180189175, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.40)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 1.07450638233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.074506382333066, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.07)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: left, reward: 0.752734973848
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 0.7527349738476103, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.75)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 1.70604881223
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 1.706048812227898, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 1.71)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: right, reward: 1.35943973232
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 1.359439732316223, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.36)
37% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 455
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (7, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0106; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0106; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0106; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: left, reward: 1.16533333775
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 1.165333337749027, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.17)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 2.08072753019
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.080727530189422, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.08)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.23317428613
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.2331742861290396, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.23)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 0.597465484574
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 0.5974654845737772, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.60)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 1.63739951427
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.63739951427153, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.64)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 2.18077331145
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.1807733114516767, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 2.18)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: left, reward: 1.09021802043
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.090218020425903, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 1.09)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.40278217223
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.40278217222886, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.40)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.6010385216
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.601038521596762, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.60)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.90218092003
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.9021809200316886, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.90)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.40998528847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.409985288474493, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.41)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 1.53341045527
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.533410455271445, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.53)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 1.59616933798
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.5961693379771689, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.60)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 1.24890120005
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.2489012000473008, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.25)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 1.48916496466
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.489164964655682, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.49)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 0.815424358035
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'forward'), 'deadline': 10, 't': 15, 'action': None, 'reward': 0.815424358034593, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 0.82)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: forward, reward: 1.60139765992
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 1.6013976599158877, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.60)
32% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 456
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (8, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0105; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0105; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.86983562697
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.8698356269662653, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.87)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.40413726789
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.40413726788944, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.40)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.13856002604
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.1385600260420503, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.14)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 0.554902535025
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 0.5549025350250903, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.55)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.68693593067
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.6869359306726155, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.69)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.2358034402
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.2358034401977067, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.24)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.27729515719
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.2772951571890208, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.28)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: 0.987298282397
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 0.9872982823971219, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 0.99)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 2.66668260946
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.666682609457995, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.67)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: 1.89562438525
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.8956243852522479, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.90)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 2.64811372715
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.64811372714965, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.65)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 0.184640741985
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.18464074198486924, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove forward instead of left. (rewarded 0.18)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.13377101307
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.1337710130702967, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.13)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: 0.616679489959
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.6166794899587444, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.62)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 0.881302649325
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.8813026493247089, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.88)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: -0.491575410448
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': -0.4915754104476173, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded -0.49)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 0.412850050858
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 0.41285005085808835, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 0.41)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: 2.09356431866
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 2.0935643186643142, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.09)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 1.42560548887
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.4256054888743297, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.43)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 0.184820435195
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.18482043519452818, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.18)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 457
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (7, 3), deadline = 35
Simulating trial. . .
epsilon = 0.0104; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0104; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.65603611555
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 35, 't': 0, 'action': None, 'reward': 1.656036115551713, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.66)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.30942570488
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 34, 't': 1, 'action': None, 'reward': 2.309425704875863, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.31)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.74976336243
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 33, 't': 2, 'action': None, 'reward': 1.749763362428031, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.75)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 0.988915619707
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 32, 't': 3, 'action': None, 'reward': 0.9889156197067213, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.99)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.24511699217
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 31, 't': 4, 'action': None, 'reward': 1.245116992174451, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.25)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.54452572881
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 30, 't': 5, 'action': None, 'reward': 1.5445257288119523, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.54)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: forward, reward: 1.04440242433
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 29, 't': 6, 'action': 'forward', 'reward': 1.0444024243272219, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent followed the waypoint forward. (rewarded 1.04)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: 1.23996817387
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 28, 't': 7, 'action': 'forward', 'reward': 1.2399681738695696, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.24)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: 2.5236980484
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 27, 't': 8, 'action': 'forward', 'reward': 2.523698048396994, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.52)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 2.19810781967
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 26, 't': 9, 'action': 'forward', 'reward': 2.198107819672488, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.20)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.33096570266
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 25, 't': 10, 'action': None, 'reward': 1.3309657026615347, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.33)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 2.06477126193
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 24, 't': 11, 'action': None, 'reward': 2.064771261929959, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.06)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 2.13055063173
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 23, 't': 12, 'action': None, 'reward': 2.1305506317302485, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.13)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: 1.30549948172
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 22, 't': 13, 'action': 'left', 'reward': 1.3054994817160779, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.31)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: 1.9849351932
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 21, 't': 14, 'action': 'forward', 'reward': 1.9849351931978103, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.98)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.71203662636
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 20, 't': 15, 'action': None, 'reward': 1.7120366263620201, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.71)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.76185642484
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 19, 't': 16, 'action': None, 'reward': 1.7618564248413278, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.76)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: forward, reward: 2.44028640961
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 18, 't': 17, 'action': 'forward', 'reward': 2.440286409606036, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.44)
49% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 458
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (3, 4), deadline = 30
Simulating trial. . .
epsilon = 0.0103; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 2.9608710551
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 2.960871055100889, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.96)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 1.89317968594
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.8931796859447123, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 1.89)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 2.92214270174
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.922142701738524, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.92)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 0.975605168176
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 0.9756051681762892, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.98)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: 2.37509118566
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 2.375091185657911, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.38)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 1.94418691918
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.9441869191775882, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.94)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 2.73088875557
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.7308887555743873, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.73)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: forward, reward: 1.62117275197
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 1.6211727519721273, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.62)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: left, reward: 2.63772750174
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': 2.6377275017448865, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 2.64)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: forward, reward: 2.03435665584
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 2.0343566558387467, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.03)
67% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 459
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (7, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0102; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0102; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 2.08933775545
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.0893377554522496, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.09)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 2.30996094899
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.309960948992032, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 2.31)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.12756635064
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.1275663506381242, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.13)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.38633557407
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.386335574071754, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.39)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.72415061178
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.7241506117841139, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.72)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.64933473933
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.6493347393312625, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.65)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: left, reward: 2.2048990984
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.2048990983993297, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 2.20)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 1.69673780492
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.6967378049235782, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.70)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 0.927614067713
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 0.9276140677133107, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.93)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: forward, reward: 0.91655184694
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.9165518469399758, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 0.92)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 460
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (7, 2), deadline = 30
Simulating trial. . .
epsilon = 0.0101; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0101; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: left, reward: 2.68770395235
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 2.6877039523496595, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.69)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: left, reward: 2.39906941048
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 29, 't': 1, 'action': 'left', 'reward': 2.3990694104835213, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent followed the waypoint left. (rewarded 2.40)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 1.03136577028
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.0313657702814698, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.03)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 2.6749489821
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.6749489820951915, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.67)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 2.00860483149
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.008604831487268, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.01)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: 1.45029084209
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.4502908420925587, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.45)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 1.26904574434
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 1.269045744335206, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.27)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: left, reward: 0.519065786547
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 0.5190657865467324, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 0.52)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 2.40536302345
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 2.4053630234514514, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.41)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 1.6340290736
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.634029073595968, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent drove forward instead of right. (rewarded 1.63)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 1.8621654696
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.8621654696019323, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.86)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 2.24155697624
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'right'), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.2415569762399006, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 2.24)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 1.10478290248
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.104782902476608, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.10)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: forward, reward: 0.809556103439
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 0.8095561034391112, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.81)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 0.993735586315
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 0.9937355863147692, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 0.99)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 0.497907393976
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 0.4979073939759161, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 0.50)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: 1.25941241129
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 1.2594124112922256, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.26)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.94810428895
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'right'), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.9481042889539602, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'right')
Agent properly idled at a red light. (rewarded 1.95)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 0.820553320031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 12, 't': 18, 'action': None, 'reward': 0.8205533200307078, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.82)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.19458690441
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 11, 't': 19, 'action': None, 'reward': 1.1945869044050366, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.19)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 2.2648046786
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 20, 'action': None, 'reward': 2.2648046786006133, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.26)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: 1.23446704237
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 9, 't': 21, 'action': 'left', 'reward': 1.234467042365785, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.23)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: 1.00366833097
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 8, 't': 22, 'action': None, 'reward': 1.0036683309727086, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.00)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: 0.492548436265
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 7, 't': 23, 'action': None, 'reward': 0.4925484362650734, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.49)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: 0.901726189695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 6, 't': 24, 'action': None, 'reward': 0.9017261896952571, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.90)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: 1.67012920377
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 5, 't': 25, 'action': None, 'reward': 1.6701292037662703, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.67)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: 2.24087127624
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 26, 'action': 'forward', 'reward': 2.240871276238617, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.24)
10% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 461
\-------------------------
Environment.reset(): Trial set up with start = (5, 7), destination = (2, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0100; alpha = 0.1000
Simulating trial. . .
epsilon = 0.0100; alpha = 0.1000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: 2.23047253951
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 2.2304725395107488, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.23)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.50547719581
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.5054771958106854, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.51)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.22461252753
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.2246125275323836, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.22)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.72106891881
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.721068918812387, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.72)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 1.59519148112
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.5951914811225663, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.60)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: 2.35613457048
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.356134570480821, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.36)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.47953631376
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.479536313764868, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.48)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.0183649466
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.0183649465994438, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.02)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.48385325748
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.4838532574792902, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.48)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.05435274562
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.054352745618265, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.05)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.24047203824
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.2404720382383734, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.24)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 1.71694961114
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.7169496111396092, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.72)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 0.146805227849
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 0.1468052278490266, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 0.15)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: 1.47884777174
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.4788477717361128, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.48)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 0.852684694525
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 0.8526846945253266, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.85)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.2966247454
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.2966247454009452, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.30)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 0.819501417539
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 0.819501417538782, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.82)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: left, reward: 1.88893434607
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 1.8889343460715713, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.89)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: 1.20711440592
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': 1.2071144059156012, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.21)
24% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 1
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (5, 4), deadline = 30
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 1.55566429274
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.5556642927364754, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.56)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: forward, reward: 1.31543991633
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 1.3154399163255395, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 1.32)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 2.25673661522
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 2.2567366152224175, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.26)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 1.97454783711
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.9745478371139127, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.97)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 0.349602107063
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 0.3496021070630734, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.35)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 1.86462416224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.8646241622442532, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.86)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 2.07284801729
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.0728480172882016, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.07)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: forward, reward: 0.703493297844
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 0.7034932978439908, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.70)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: left, reward: 2.20377479219
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': 2.203774792194036, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.20)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 1.05244982807
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.0524498280739465, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.05)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 1.90788665868
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.9078866586784748, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.91)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: 2.20762630833
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 2.2076263083303833, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.21)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.28288994368
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.2828899436763357, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.28)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.38388395892
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 17, 't': 13, 'action': None, 'reward': 2.383883958921805, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.38)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.08110031234
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.0811003123382774, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.08)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.57801317124
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': None, 'reward': 2.5780131712393417, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.58)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: 2.72471356761
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': 2.72471356761102, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.72)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: left, reward: 0.946239586344
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 13, 't': 17, 'action': 'left', 'reward': 0.9462395863438153, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 0.95)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: forward, reward: 0.952628603166
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 12, 't': 18, 'action': 'forward', 'reward': 0.9526286031661635, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 0.95)
37% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 2
\-------------------------
Environment.reset(): Trial set up with start = (5, 7), destination = (6, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.02180988806
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.0218098880601543, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.02)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: right, reward: 0.588846346626
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.5888463466258921, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent drove right instead of left. (rewarded 0.59)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: forward, reward: 1.18181671599
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.18181671598684, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent drove forward instead of right. (rewarded 1.18)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: right, reward: 2.73585811415
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 2.7358581141450804, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.74)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: left, reward: 1.04738923409
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.0473892340853412, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.05)
75% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 3
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (5, 6), deadline = 30
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: left, reward: 2.99055758131
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 2.990557581311214, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.99)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 1.3780070766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.3780070765985757, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.38)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 2.07221777819
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.072217778191807, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.07)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 2.78731657083
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.78731657082857, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.79)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 2.46254321288
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.462543212882472, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.46)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: forward, reward: 1.60815977752
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.6081597775150407, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.61)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: forward, reward: 1.9532090501
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 1.9532090500974468, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.95)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: right, reward: 0.99023808994
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 0.9902380899398577, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.99)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: None, reward: 2.56054091293
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.5605409129280456, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.56)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: None, reward: 0.941832353278
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 9, 'action': None, 'reward': 0.9418323532776149, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.94)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: forward, reward: 2.72092465463
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 2.7209246546287096, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.72)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: forward, reward: 1.69193468972
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.6919346897223178, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.69)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 4
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (5, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 2.56969189965
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.569691899650736, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.57)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 2.7501535808
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 2.750153580801332, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.75)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 2.89971795212
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 2.8997179521208922, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.90)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 2.00449612795
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.0044961279478066, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.00)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 1.93989534243
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.9398953424307153, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.94)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: forward, reward: 1.06334858451
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.0633485845087598, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.06)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.79873438719
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.79873438719318, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.80)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: right, reward: 1.61850770219
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'right', None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.6185077021906833, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', None)
Agent followed the waypoint right. (rewarded 1.62)
68% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 5
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (8, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 2.9205319123
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.9205319123024553, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.92)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 2.4621556543
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.462155654296647, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.46)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.50169896454
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.5016989645388523, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.50)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.43987601221
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.4398760122053063, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.44)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 1.31390588825
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.3139058882509917, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.31)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 0.242323791239
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.24232379123946135, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 0.24)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 2.17617442791
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.176174427909042, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.18)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 2.43955865185
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.4395586518508683, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.44)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.49657626946
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.4965762694573874, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.50)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.28063779704
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.2806377970387184, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.28)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 2.52708090108
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.527080901084081, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.53)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: 1.54159729576
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 1.5415972957624815, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.54)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 2.06460914198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 2.064609141977039, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 2.06)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: 1.89209919843
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.8920991984343731, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.89)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: forward, reward: 0.854265382425
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 0.8542653824247279, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 0.85)
25% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 6
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (6, 2), deadline = 35
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.48966980457
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 35, 't': 0, 'action': None, 'reward': 1.4896698045655041, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.49)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.27231903454
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 34, 't': 1, 'action': None, 'reward': 1.2723190345425044, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.27)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.67150890846
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 33, 't': 2, 'action': None, 'reward': 1.6715089084631642, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.67)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 2.12426933772
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 32, 't': 3, 'action': 'right', 'reward': 2.1242693377179287, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.12)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 1.65321398976
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 31, 't': 4, 'action': None, 'reward': 1.6532139897643305, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.65)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 2.43343883479
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 30, 't': 5, 'action': 'right', 'reward': 2.4334388347909632, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.43)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: left, reward: 0.880672720248
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 29, 't': 6, 'action': 'left', 'reward': 0.8806727202481859, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.88)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 2.54060562922
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 28, 't': 7, 'action': None, 'reward': 2.540605629216893, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.54)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 0.923695581537
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 27, 't': 8, 'action': 'right', 'reward': 0.923695581536778, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.92)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 2.06469334556
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 26, 't': 9, 'action': 'forward', 'reward': 2.064693345555452, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.06)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.21077936267
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 25, 't': 10, 'action': None, 'reward': 2.2107793626664356, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.21)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 2.13838341581
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 24, 't': 11, 'action': 'forward', 'reward': 2.1383834158123274, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.14)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.272121416
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', None), 'deadline': 23, 't': 12, 'action': None, 'reward': 1.2721214160033747, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 1.27)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: 2.02664581042
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 22, 't': 13, 'action': 'left', 'reward': 2.02664581041567, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.03)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 7
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (2, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: 1.87166947904
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.8716694790375161, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.87)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 1.18636925911
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.1863692591098576, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove right instead of forward. (rewarded 1.19)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 1.5229587831
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.5229587831007276, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.52)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: forward, reward: 0.061207986873
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 0.061207986873012765, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.06)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 0.893302183063
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.8933021830627424, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent drove right instead of left. (rewarded 0.89)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 1.28523806459
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.2852380645920054, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.29)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 2.73248711077
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.7324871107710487, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.73)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: right, reward: 0.294351198135
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.29435119813450594, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.29)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 0.832346536621
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.8323465366207626, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 0.83)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 1.86542420234
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.8654242023383623, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.87)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 1.24629491182
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.2462949118239908, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.25)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 2.12972803734
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.1297280373412133, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.13)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 2.06787494984
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.0678749498431, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.07)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 1.68899422542
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.688994225420425, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.69)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 1.03252852321
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.0325285232086574, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.03)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 2.22624511995
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'right'), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.226245119953393, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'right')
Agent properly idled at a red light. (rewarded 2.23)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 0.948337018386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.9483370183855782, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.95)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 1.08377202773
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 1.08377202772679, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.08)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 1.05914011138
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 1.0591401113766152, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.06)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 1.39879635273
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 1.3987963527338196, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.40)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 8
\-------------------------
Environment.reset(): Trial set up with start = (2, 7), destination = (7, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: forward, reward: -10.9974430142
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'left', 'left'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -10.99744301415674, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -11.00)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: right, reward: 2.90724315215
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.9072431521538618, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.91)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 1.36365045538
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'forward'), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 1.363650455376559, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.36)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 1.9874054551
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.9874054551006042, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.99)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.04029069211
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.040290692109271, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.04)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 2.46857395615
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 2.4685739561467117, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 2.47)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 0.624834742031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.6248347420309582, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.62)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: left, reward: 2.18966231787
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 2.1896623178653285, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 2.19)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: forward, reward: 1.24669716278
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 1.246697162781738, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 1.25)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: left, reward: 1.0067574344
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.0067574343980985, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.01)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 2.46486060521
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.464860605205822, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.46)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: 1.50266945073
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.502669450731664, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.50)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: left, reward: 1.90959137689
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 12, 'action': 'left', 'reward': 1.9095913768948227, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.91)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: left, reward: 1.76329431073
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 1.763294310727983, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.76)
44% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 9
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (7, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: forward, reward: 1.31067172676
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'right'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.310671726761409, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'right')
Agent drove forward instead of right. (rewarded 1.31)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 2.0680772247
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.068077224701893, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 2.07)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 0.0307785542862
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.030778554286150173, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 0.03)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 1.89126855654
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.8912685565370526, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.89)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 2.59010915787
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.5901091578667748, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.59)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: 2.39443293863
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.3944329386257013, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.39)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: 2.0931009537
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 2.093100953698655, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.09)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: left, reward: 0.256059880597
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 0.2560598805971852, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.26)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: right, reward: 2.525825986
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.5258259859998105, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 2.53)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: left, reward: 0.00799333397668
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 0.007993333976682004, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 0.01)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.01233939988
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.012339399881983, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.01)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: right, reward: 1.87486453727
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.8748645372675208, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.87)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 1.15540868428
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.1554086842792377, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.16)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: None, reward: 2.17346298275
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 2.173462982748558, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.17)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 0.966977791634
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 0.9669777916341136, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.97)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.42770132889
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.4277013288907225, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.43)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 1.27064457542
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.2706445754159519, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.27)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 0.574446665166
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.5744466651662914, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.57)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.37112277854
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.3711227785418036, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 1.37)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 0.486156581965
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.48615658196461853, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.49)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 10
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (7, 7), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.17532896422
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.1753289642248244, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.18)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.05968263707
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.0596826370725534, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.06)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.79362455994
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.793624559937854, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.79)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.82828831241
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.8282883124147404, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.83)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.07020688461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.070206884611484, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.07)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.77944818442
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.7794481844238392, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.78)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: 0.458378403784
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 0.45837840378430517, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.46)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: 1.47164926458
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 1.4716492645811432, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.47)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: right, reward: 0.500403354232
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'right'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.5004033542316545, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'right')
Agent drove right instead of forward. (rewarded 0.50)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 2.06898132438
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.068981324380429, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.07)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 1.56863495915
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.5686349591544606, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.57)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: 0.539855945184
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 0.5398559451837531, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.54)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: -0.0827609789203
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': -0.08276097892034606, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded -0.08)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 2.19581578425
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 2.1958157842459283, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.20)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 2.21375746766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.2137574676624574, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.21)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 1.94796243376
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.9479624337646373, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.95)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 0.361165823312
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 0.36116582331186764, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', None)
Agent drove forward instead of left. (rewarded 0.36)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: left, reward: 1.50284327798
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 1.502843277982511, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove left instead of forward. (rewarded 1.50)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: 0.537912773745
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 0.5379127737448526, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.54)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 2.40264653802
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 19, 'action': None, 'reward': 2.4026465380223057, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.40)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: 0.454381492202
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 0.4543814922024769, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.45)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: 1.1069953381
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 1.1069953381040631, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.11)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: right, reward: 1.25312654753
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 1.2531265475332654, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.25)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 1.09298089765
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.0929808976489064, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.09)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 1.88899962265
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.888999622650218, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.89)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
Simulation ended. . .
<matplotlib.figure.Figure at 0x90155c0>
In [27]:
# self.epsilon = math.exp(-(self.a*self.t))
# epsilon-tolerance = 0.01 alpha = 0.1
# 训练次数：475
vs.plot_trials('sim_improved-learning.csv')
In [30]:
%run smartcab\agent.py
/-------------------------
| Training trial 1
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (8, 6), deadline = 25
Simulating trial. . .
epsilon = 0.9900; alpha = 0.5000
Simulating trial. . .
epsilon = 0.9900; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: forward, reward: 0.460738205148
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 0.4607382051476161, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 0.46)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 1.21731624485
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.217316244845887, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.22)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: -4.313425734
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': -4.313425734002182, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.31)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: 2.53019340241
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.530193402409763, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', None)
Agent followed the waypoint forward. (rewarded 2.53)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: -10.833170812
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -10.833170811951579, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', None)
Agent attempted driving forward through a red light. (rewarded -10.83)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: right, reward: 0.566878380496
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.5668783804958072, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.57)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: -4.54822964199
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'left', 'right'), 'deadline': 19, 't': 6, 'action': None, 'reward': -4.548229641988896, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.55)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 1.18639581136
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.1863958113569133, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.19)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: 2.23041903831
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.2304190383083977, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.23)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 0.569841510531
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.5698415105305976, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent drove right instead of left. (rewarded 0.57)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: 1.26541525323
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.2654152532348295, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.27)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: -0.140092555311
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': -0.14009255531094877, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove left instead of right. (rewarded -0.14)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: forward, reward: -9.09739065072
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': -9.097390650719577, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.10)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: left, reward: 0.146357628637
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 0.14635762863725876, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.15)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: forward, reward: 1.25434218126
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 1.254342181263524, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove forward instead of left. (rewarded 1.25)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: left, reward: 1.06082455153
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 1.0608245515305863, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.06)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: left, reward: 1.24454220467
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 1.244542204671268, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 1.24)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: left, reward: -40.7431484995
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': -40.74314849949057, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.74)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: None, reward: 1.38014118457
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.3801411845748373, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.38)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: left, reward: 0.368727782896
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 0.3687277828956953, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent drove left instead of right. (rewarded 0.37)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.60258862174
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.6025886217376746, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.60)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.25925026143
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 4, 't': 21, 'action': None, 'reward': 1.259250261432728, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.26)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: 0.940592879521
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.9405928795213359, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 0.94)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: left, reward: -19.8115715876
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, 'right', 'forward'), 'deadline': 2, 't': 23, 'action': 'left', 'reward': -19.811571587619817, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.81)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 0.446738466605
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.44673846660495653, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.45)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 2
\-------------------------
Environment.reset(): Trial set up with start = (3, 2), destination = (6, 6), deadline = 25
Simulating trial. . .
epsilon = 0.9802; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 1.81345346121
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.8134534612073647, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.81)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: 1.83945989359
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.839459893589828, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.84)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: forward, reward: 2.34528033294
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 2.3452803329415266, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.35)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: 2.09864997991
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.098649979906614, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.10)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: -9.71216389001
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': -9.712163890008938, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.71)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: -5.83541458826
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': -5.835414588256095, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.84)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: -5.47124479402
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': -5.471244794023065, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.47)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 0.682798770578
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.6827987705776326, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.68)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.56725784285
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.5672578428526194, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.57)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 1.6696412448
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.6696412448017381, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.67)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: -5.61623100731
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 15, 't': 10, 'action': None, 'reward': -5.616231007307043, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.62)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 1.59510798555
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.5951079855532089, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.60)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 1.08199926488
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.0819992648800016, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.08)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 1.15821147051
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 1.1582114705147488, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent followed the waypoint forward. (rewarded 1.16)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: left, reward: 1.53275778141
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 1.5327577814147197, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove left instead of forward. (rewarded 1.53)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 1.11385836178
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.1138583617823294, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.11)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 2.45350393405
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 9, 't': 16, 'action': None, 'reward': 2.4535039340503637, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.45)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 2.20104529082
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', 'left'), 'deadline': 8, 't': 17, 'action': None, 'reward': 2.201045290822614, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.20)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: left, reward: -10.5098060152
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 7, 't': 18, 'action': 'left', 'reward': -10.50980601521312, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent attempted driving left through a red light. (rewarded -10.51)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: left, reward: 0.723258166328
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'left'), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 0.7232581663281862, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'left')
Agent drove left instead of right. (rewarded 0.72)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 0.78357219078
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'right', None), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 0.7835721907797439, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'right', None)
Agent drove right instead of forward. (rewarded 0.78)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.37260869929
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 4, 't': 21, 'action': None, 'reward': 1.3726086992891204, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.37)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: -5.12520506819
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 3, 't': 22, 'action': None, 'reward': -5.125205068192225, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.13)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: -0.388884614942
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 2, 't': 23, 'action': 'right', 'reward': -0.3888846149415621, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded -0.39)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 0.528663223404
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 1, 't': 24, 'action': 'right', 'reward': 0.5286632234039919, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.53)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 3
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (1, 4), deadline = 20
Simulating trial. . .
epsilon = 0.9704; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: -4.28908539603
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': -4.289085396034275, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.29)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: forward, reward: 0.689668643527
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 0.6896686435272509, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 0.69)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: None, reward: -4.08039343603
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': -4.080393436025586, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.08)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: right, reward: -20.3182754404
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': -20.318275440372602, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.32)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: None, reward: 1.94239193285
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.9423919328542456, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.94)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: right, reward: 0.475870141477
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.47587014147734985, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 0.48)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 0.622689358397
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 0.6226893583971558, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.62)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: right, reward: 2.65375959337
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 2.653759593371076, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.65)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: right, reward: 1.64977203764
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.6497720376422051, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 1.65)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: left, reward: -40.2811678624
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': -40.28116786239216, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.28)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: left, reward: -9.28043627101
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -9.280436271007764, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.28)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: None, reward: -5.11846627291
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'forward'), 'deadline': 9, 't': 11, 'action': None, 'reward': -5.118466272907844, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.12)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: left, reward: -20.7217490103
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'forward'), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -20.72174901029291, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.72)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: left, reward: -10.2264183862
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': -10.22641838622427, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent attempted driving left through a red light. (rewarded -10.23)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: left, reward: -10.625426693
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -10.625426693045704, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.63)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: left, reward: 0.9130243652
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 0.9130243651996544, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove left instead of forward. (rewarded 0.91)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: right, reward: 1.22585103711
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.2258510371066176, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', None)
Agent followed the waypoint right. (rewarded 1.23)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: right, reward: -19.4222013376
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 3, 't': 17, 'action': 'right', 'reward': -19.42220133757357, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.42)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: -0.320104644959
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'forward'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': -0.3201046449589877, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'forward')
Agent drove right instead of forward. (rewarded -0.32)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: -4.81477005343
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': -4.814770053425006, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.81)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 4
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (1, 6), deadline = 20
Simulating trial. . .
epsilon = 0.9608; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: 2.01970730015
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.019707300145706, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.02)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 1.6569745098
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.6569745097976556, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.66)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: forward, reward: -10.6932970374
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': -10.693297037360383, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.69)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 2.72207173261
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.7220717326117163, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.72)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: left, reward: -10.9171432951
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -10.917143295073968, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.92)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: 1.53893888745
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.5389388874514378, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.54)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: -9.54369032126
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -9.543690321262325, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.54)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: -19.6464758707
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': -19.646475870666524, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.65)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 1.50137979194
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.5013797919440575, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.50)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: 1.16756917391
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.1675691739082166, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.17)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 0.930798796308
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 0.9307987963082885, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded 0.93)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: -9.57433725441
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': -9.574337254413335, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.57)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 0.786596708865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.7865967088654497, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.79)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: -0.214649767744
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': -0.21464976774393885, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded -0.21)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 1.31259105068
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', 'forward'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.3125910506833494, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', 'forward')
Agent followed the waypoint right. (rewarded 1.31)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: -10.9873236924
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -10.987323692403475, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.99)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: -10.0716389758
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': 'left', 'reward': -10.071638975788842, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.07)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: -5.45702526865
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'right'), 'deadline': 3, 't': 17, 'action': None, 'reward': -5.457025268645586, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.46)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: -19.3039893849
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'right'), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -19.303989384877088, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.30)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: left, reward: -0.753853146401
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': -0.7538531464005532, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded -0.75)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 5
\-------------------------
Environment.reset(): Trial set up with start = (7, 6), destination = (4, 3), deadline = 30
Simulating trial. . .
epsilon = 0.9512; alpha = 0.5000
Simulating trial. . .
epsilon = 0.9512; alpha = 0.5000
Simulating trial. . .
epsilon = 0.9512; alpha = 0.5000
Simulating trial. . .
epsilon = 0.9512; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.01674324567
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.0167432456688041, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.02)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.13970709913
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.1397070991278937, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.14)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 1.1943202377
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 1.1943202376993982, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.19)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: left, reward: 1.33818541501
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 27, 't': 3, 'action': 'left', 'reward': 1.3381854150117725, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 1.34)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: -5.19818178735
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': -5.1981817873516825, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.20)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: -9.17991704498
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': -9.179917044981496, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -9.18)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: -39.6354570846
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': -39.635457084635625, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.64)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: left, reward: -20.5912317846
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 23, 't': 7, 'action': 'left', 'reward': -20.591231784552168, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.59)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: -4.74801949422
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': -4.74801949422449, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.75)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: left, reward: -10.6639770531
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': -10.663977053065777, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.66)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: right, reward: 1.16764791115
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.1676479111465066, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.17)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: left, reward: 1.46624360962
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 1.4662436096208111, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 1.47)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: -4.54170692259
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': -4.541706922593195, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.54)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: -19.0376829621
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 17, 't': 13, 'action': 'right', 'reward': -19.037682962076136, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.04)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.72303379309
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.7230337930947974, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.72)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: left, reward: -39.1475210484
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 15, 't': 15, 'action': 'left', 'reward': -39.147521048388654, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.15)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: 0.596936830283
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 0.5969368302832485, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.60)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 1.1359087597
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 1.1359087597040802, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove right instead of left. (rewarded 1.14)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: -10.777694788
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 12, 't': 18, 'action': 'left', 'reward': -10.77769478798034, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.78)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: right, reward: 1.11341093859
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 1.1134109385896238, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.11)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: -0.312050365864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 10, 't': 20, 'action': None, 'reward': -0.3120503658637124, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.31)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: forward, reward: -9.30928617644
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': -9.309286176436053, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.31)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 2.36941053232
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 2.369410532315797, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 2.37)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: -0.199996287123
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 7, 't': 23, 'action': 'right', 'reward': -0.19999628712313722, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded -0.20)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 0.648570419977
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 6, 't': 24, 'action': None, 'reward': 0.6485704199770961, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.65)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 0.894289837374
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 0.8942898373743817, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.89)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: left, reward: 1.06878405873
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 4, 't': 26, 'action': 'left', 'reward': 1.06878405873265, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 1.07)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: -0.667434664054
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 3, 't': 27, 'action': 'right', 'reward': -0.6674346640535225, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove right instead of left. (rewarded -0.67)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: -9.4157532118
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 2, 't': 28, 'action': 'left', 'reward': -9.415753211796035, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.42)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: -40.7739762953
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 1, 't': 29, 'action': 'left', 'reward': -40.7739762952855, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.77)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 6
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (3, 3), deadline = 25
Simulating trial. . .
epsilon = 0.9418; alpha = 0.5000
Simulating trial. . .
epsilon = 0.9418; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: left, reward: -9.07568879644
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': -9.07568879644129, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -9.08)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 1.45574776312
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.4557477631170657, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.46)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 2.53057392031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 2.5305739203112623, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent followed the waypoint right. (rewarded 2.53)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: -5.50310421632
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': -5.5031042163233215, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.50)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: left, reward: -9.57389960009
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': -9.573899600090247, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -9.57)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 2.70355361668
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.7035536166832435, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.70)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: left, reward: -10.3032669817
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': -10.303266981743603, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.30)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 1.6779894166
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.6779894165970433, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.68)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: forward, reward: 0.277163145868
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 0.2771631458684466, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded 0.28)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 2.42791455091
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.427914550909947, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.43)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 1.4328973292
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.432897329201985, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.43)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 1.30573846187
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.3057384618706358, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.31)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: -9.46958930036
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 13, 't': 12, 'action': 'left', 'reward': -9.46958930035943, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent attempted driving left through a red light. (rewarded -9.47)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 2.1984493529
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 2.1984493529036504, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent followed the waypoint right. (rewarded 2.20)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: left, reward: -20.4677560541
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'right'), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -20.46775605407292, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.47)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: left, reward: -10.7945435918
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 10, 't': 15, 'action': 'left', 'reward': -10.794543591844358, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.79)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 1.17129424796
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 1.1712942479648114, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.17)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: left, reward: -9.3049169156
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': -9.304916915597147, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.30)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: -20.3700761182
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': -20.37007611816645, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.37)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: -4.91294315418
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 6, 't': 19, 'action': None, 'reward': -4.9129431541774204, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.91)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: 1.75345388035
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 20, 'action': 'left', 'reward': 1.7534538803473991, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.75)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: 2.15409799011
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 2.1540979901077213, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.15)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.10632172478
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 3, 't': 22, 'action': None, 'reward': 2.106321724783812, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 2.11)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.14141456563
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.1414145656250232, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.14)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: -10.9395610396
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': -10.939561039632475, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.94)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 7
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (5, 5), deadline = 30
Simulating trial. . .
epsilon = 0.9324; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 1.73237273318
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.7323727331806822, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 1.73)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: -4.20673886359
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': -4.20673886359288, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.21)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 1.23168026677
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 1.231680266765918, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.23)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 2.76654429586
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 2.7665442958646644, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent followed the waypoint forward. (rewarded 2.77)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 0.580963156073
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 0.5809631560728786, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.58)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: forward, reward: -40.0045703652
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'right'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': -40.004570365188194, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'right')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.00)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 1.19907446332
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 1.1990744633237433, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.20)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: -10.4246783843
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': -10.424678384288349, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.42)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 0.259865133539
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 0.25986513353945406, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.26)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 2.43878116602
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 2.4387811660225056, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 2.44)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: 1.57255127105
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 1.5725512710487168, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.57)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: 1.06857103116
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.0685710311647993, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.07)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: left, reward: -10.6042591664
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': 'left', 'reward': -10.60425916635203, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.60)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: 0.571727734015
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'right'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 0.5717277340154621, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'right')
Agent drove forward instead of right. (rewarded 0.57)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: -10.5468303188
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': -10.546830318814607, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -10.55)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 1.63224271765
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'forward'), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.6322427176476688, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.63)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 1.05023614815
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 1.0502361481549738, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.05)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 0.76955071645
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 13, 't': 17, 'action': None, 'reward': 0.7695507164496302, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.77)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: -5.18431771765
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'left', 'right'), 'deadline': 12, 't': 18, 'action': None, 'reward': -5.184317717654261, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.18)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: 0.0662453591679
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': 0.06624535916787744, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 0.07)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: forward, reward: 1.0229553091
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 10, 't': 20, 'action': 'forward', 'reward': 1.0229553090958352, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent drove forward instead of right. (rewarded 1.02)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: -5.51524938278
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 9, 't': 21, 'action': None, 'reward': -5.5152493827809685, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.52)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: right, reward: 1.99684797214
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 1.9968479721352623, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.00)
23% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 8
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (1, 3), deadline = 30
Simulating trial. . .
epsilon = 0.9231; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: left, reward: -20.6558356795
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 30, 't': 0, 'action': 'left', 'reward': -20.65583567953008, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.66)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: right, reward: 2.80983138971
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 2.8098313897069813, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.81)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: -9.46015841622
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': 'left', 'reward': -9.460158416218771, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.46)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.1604983198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.1604983198034065, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.16)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: -10.8652351596
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 26, 't': 4, 'action': 'left', 'reward': -10.865235159587044, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.87)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: left, reward: 0.773089115075
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 0.7730891150751728, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 0.77)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 1.92578342134
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 1.9257834213406206, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.93)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: -10.7185721087
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': -10.71857210870742, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.72)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: -10.3900815536
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': -10.390081553638435, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.39)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 1.23882784901
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.238827849005797, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.24)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: right, reward: 0.889888737973
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 0.8898887379726166, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.89)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.74775762025
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.7477576202526657, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.75)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: forward, reward: -9.57163544496
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': -9.571635444961668, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -9.57)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: right, reward: 2.12727323658
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 2.1272732365755926, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.13)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 0.67970210914
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': 0.6797021091396225, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 0.68)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.85196869966
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'right', None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.8519686996577314, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'right', None)
Agent followed the waypoint right. (rewarded 1.85)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: left, reward: -39.9751445715
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': -39.97514457148239, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.98)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: 1.94061685918
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.9406168591813158, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.94)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 1.44522655164
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 1.445226551640612, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.45)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 0.466201872769
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 0.4662018727693601, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove right instead of forward. (rewarded 0.47)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: -5.16623841112
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 10, 't': 20, 'action': None, 'reward': -5.166238411120737, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.17)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: -19.1250011592
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 9, 't': 21, 'action': 'right', 'reward': -19.125001159191303, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.13)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: forward, reward: -10.7166226276
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': -10.716622627621225, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -10.72)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: -4.83468677352
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 23, 'action': None, 'reward': -4.834686773515664, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.83)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: forward, reward: -0.388204871668
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 6, 't': 24, 'action': 'forward', 'reward': -0.38820487166750794, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded -0.39)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: forward, reward: -9.55889504164
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 5, 't': 25, 'action': 'forward', 'reward': -9.558895041640799, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.56)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: left, reward: -10.7873117963
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 4, 't': 26, 'action': 'left', 'reward': -10.787311796327606, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.79)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: left, reward: 0.77006350018
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 3, 't': 27, 'action': 'left', 'reward': 0.7700635001797294, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 0.77)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: left, reward: -40.2485243471
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'forward'), 'deadline': 2, 't': 28, 'action': 'left', 'reward': -40.24852434705771, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.25)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 0.404105554254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 1, 't': 29, 'action': None, 'reward': 0.4041055542543146, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 0.40)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 9
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (8, 6), deadline = 30
Simulating trial. . .
epsilon = 0.9139; alpha = 0.5000
Simulating trial. . .
epsilon = 0.9139; alpha = 0.5000
Simulating trial. . .
epsilon = 0.9139; alpha = 0.5000
Simulating trial. . .
epsilon = 0.9139; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: left, reward: -39.1777765926
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'left'), 'deadline': 30, 't': 0, 'action': 'left', 'reward': -39.17777659255364, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.18)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: -10.771753039
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': -10.771753039030111, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.77)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 2.37332698917
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 2.3733269891702045, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.37)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: -20.09343851
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'forward'), 'deadline': 27, 't': 3, 'action': 'left', 'reward': -20.093438510019748, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.09)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: left, reward: 0.686758956887
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 0.6867589568873417, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove left instead of forward. (rewarded 0.69)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 1.44540077764
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'right', None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.445400777643767, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', None)
Agent followed the waypoint right. (rewarded 1.45)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.60916936228
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.609169362281735, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.61)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: left, reward: 1.69606168006
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 1.6960616800564865, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.70)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: right, reward: 1.14184595314
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 1.1418459531428486, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', None)
Agent followed the waypoint right. (rewarded 1.14)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 1.13656361053
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 1.1365636105299137, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent drove right instead of left. (rewarded 1.14)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: left, reward: 0.873053848922
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 0.8730538489218176, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.87)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: -40.8010594306
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': -40.80105943055743, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.80)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: left, reward: -9.35296767886
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': 'left', 'reward': -9.352967678858386, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.35)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: 1.07874022054
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.078740220537231, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.08)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: -9.33576915362
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': -9.335769153615702, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent attempted driving forward through a red light. (rewarded -9.34)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: left, reward: 2.47106132474
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 15, 't': 15, 'action': 'left', 'reward': 2.4710613247356066, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 2.47)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: forward, reward: 1.23324779562
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': 1.233247795622385, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.23)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: forward, reward: -40.2863662107
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': -40.28636621074166, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.29)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: right, reward: -0.00538963809897
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 12, 't': 18, 'action': 'right', 'reward': -0.00538963809896531, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent drove right instead of left. (rewarded -0.01)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: -5.07572723933
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'forward'), 'deadline': 11, 't': 19, 'action': None, 'reward': -5.075727239332546, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.08)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: -4.97735938509
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'forward'), 'deadline': 10, 't': 20, 'action': None, 'reward': -4.977359385091459, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.98)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: left, reward: -39.0880659291
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 9, 't': 21, 'action': 'left', 'reward': -39.0880659291263, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.09)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 1.59853955018
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 8, 't': 22, 'action': None, 'reward': 1.5985395501764528, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.60)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: left, reward: -9.94273962641
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 7, 't': 23, 'action': 'left', 'reward': -9.94273962640758, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent attempted driving left through a red light. (rewarded -9.94)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: left, reward: -10.5448777159
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': 'left', 'reward': -10.544877715882144, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.54)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: left, reward: -20.2455483121
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 5, 't': 25, 'action': 'left', 'reward': -20.245548312091824, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.25)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: left, reward: -0.336712067034
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 4, 't': 26, 'action': 'left', 'reward': -0.3367120670342598, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove left instead of right. (rewarded -0.34)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: left, reward: -9.45678185795
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 3, 't': 27, 'action': 'left', 'reward': -9.456781857951164, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.46)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: right, reward: -0.712910253064
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 28, 'action': 'right', 'reward': -0.7129102530636359, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded -0.71)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: left, reward: 0.0680736551452
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 1, 't': 29, 'action': 'left', 'reward': 0.06807365514522301, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 0.07)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 10
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (4, 4), deadline = 30
Simulating trial. . .
epsilon = 0.9048; alpha = 0.5000
Simulating trial. . .
epsilon = 0.9048; alpha = 0.5000
Simulating trial. . .
epsilon = 0.9048; alpha = 0.5000
Simulating trial. . .
epsilon = 0.9048; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: -5.89545441886
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'right'), 'deadline': 30, 't': 0, 'action': None, 'reward': -5.8954544188601705, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.90)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: left, reward: 1.14188656182
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', 'left'), 'deadline': 29, 't': 1, 'action': 'left', 'reward': 1.141886561821485, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', 'left')
Agent followed the waypoint left. (rewarded 1.14)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 1.10495414798
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 1.10495414798262, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.10)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: -5.44824691453
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 27, 't': 3, 'action': None, 'reward': -5.448246914528516, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.45)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 0.0304294991051
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 0.030429499105105617, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove right instead of forward. (rewarded 0.03)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 2.20096005483
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.200960054830108, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.20)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 0.433441047225
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 0.43344104722497734, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 0.43)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: left, reward: 2.1778066801
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 2.1778066800987994, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 2.18)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: -5.26365903656
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': -5.26365903655662, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.26)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: left, reward: 2.83721994081
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 2.8372199408071523, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.84)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 0.666165211764
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 0.6661652117643113, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove right instead of forward. (rewarded 0.67)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: forward, reward: 1.64458939604
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'left'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.644589396037814, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'left')
Agent drove forward instead of left. (rewarded 1.64)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: -0.165887450954
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 18, 't': 12, 'action': 'right', 'reward': -0.16588745095397062, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove right instead of left. (rewarded -0.17)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: -20.6741236522
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'forward'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': -20.674123652166585, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.67)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: -9.6824309414
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': -9.682430941396982, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -9.68)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: -9.74365780032
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': -9.74365780032368, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.74)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: -39.8742841317
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 14, 't': 16, 'action': 'left', 'reward': -39.87428413174924, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.87)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: -20.5529783649
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 13, 't': 17, 'action': 'left', 'reward': -20.552978364867105, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.55)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: -20.2290726542
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 12, 't': 18, 'action': 'left', 'reward': -20.229072654183746, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.23)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: -4.22801787752
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 11, 't': 19, 'action': None, 'reward': -4.2280178775209105, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.23)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: -5.72665157166
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 10, 't': 20, 'action': None, 'reward': -5.726651571662873, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.73)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.11845017385
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 9, 't': 21, 'action': None, 'reward': 1.1184501738486774, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.12)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: -9.46276778169
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 22, 'action': 'left', 'reward': -9.462767781686802, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.46)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: -10.1814981191
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 7, 't': 23, 'action': 'left', 'reward': -10.181498119097666, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent attempted driving left through a red light. (rewarded -10.18)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: -10.8334494355
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': 'forward', 'reward': -10.83344943550854, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.83)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: -19.609800431
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 5, 't': 25, 'action': 'left', 'reward': -19.609800431046025, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.61)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: -5.80019822769
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 4, 't': 26, 'action': None, 'reward': -5.800198227690039, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.80)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: -5.19894349429
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 27, 'action': None, 'reward': -5.198943494286821, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.20)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 1.36675019093
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 2, 't': 28, 'action': 'right', 'reward': 1.3667501909261837, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.37)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: -0.526322662051
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 1, 't': 29, 'action': 'forward', 'reward': -0.526322662051401, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded -0.53)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 11
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (8, 4), deadline = 20
Simulating trial. . .
epsilon = 0.8958; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 1.22648581705
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'right'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.2264858170538504, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'right')
Agent properly idled at a red light. (rewarded 1.23)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: right, reward: 1.83053641368
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.830536413675207, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.83)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: forward, reward: 0.991826438762
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 0.9918264387617074, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent drove forward instead of right. (rewarded 0.99)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: left, reward: -10.7838300098
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -10.783830009843125, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.78)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 1.30343635566
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.3034363556634652, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.30)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: right, reward: 1.65658958394
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.656589583937959, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.66)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: -5.03533217311
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': -5.035332173110955, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.04)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.84254910183
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.8425491018302789, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.84)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: right, reward: -20.7657761225
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': -20.76577612248414, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.77)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: left, reward: -40.9680932556
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': -40.96809325563201, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.97)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: left, reward: 1.13714292638
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.1371429263765256, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', 'left')
Agent drove left instead of forward. (rewarded 1.14)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 1.71466983817
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.7146698381709147, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.71)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 0.313847800862
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.3138478008619441, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.31)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: 0.495621990712
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.49562199071195934, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.50)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: -40.7178439731
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': -40.71784397305726, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.72)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: left, reward: -9.98796748462
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -9.987967484618245, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -9.99)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: left, reward: -20.7401025115
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 4, 't': 16, 'action': 'left', 'reward': -20.74010251151973, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.74)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: -5.46900202896
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 3, 't': 17, 'action': None, 'reward': -5.469002028961077, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.47)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: right, reward: -0.420900890254
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': -0.4209008902539295, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded -0.42)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 0.284768545155
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.28476854515466854, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove right instead of left. (rewarded 0.28)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 12
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (5, 3), deadline = 25
Simulating trial. . .
epsilon = 0.8869; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: -9.7255879996
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -9.72558799959576, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.73)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: -10.4514670615
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -10.451467061478244, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent attempted driving left through a red light. (rewarded -10.45)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.17866669175
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.178666691747115, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.18)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 1.32714766956
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.3271476695587805, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 1.33)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: 1.18125796367
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 1.181257963667826, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 1.18)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.48728787873
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'left'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.4872878787293693, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 1.49)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 1.34093614703
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.3409361470329626, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.34)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: forward, reward: 0.0745382386771
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 0.0745382386771134, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded 0.07)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 1.20163629032
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.2016362903245705, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.20)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 0.97716874165
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.9771687416501986, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 0.98)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 0.798689483256
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.7986894832563257, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded 0.80)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 1.28084032356
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.2808403235599497, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.28)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: -39.6513434534
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', None), 'deadline': 13, 't': 12, 'action': 'left', 'reward': -39.65134345339271, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.65)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: -5.03590798664
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': -5.035907986642574, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.04)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 0.792617634325
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 0.7926176343251892, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 0.79)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 1.34725208997
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.347252089969123, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.35)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 2.32958493789
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 9, 't': 16, 'action': None, 'reward': 2.32958493789235, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.33)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 1.76810363475
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.768103634746796, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 1.77)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 1.46454461563
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 1.4645446156277564, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 1.46)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 0.0472181166344
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', None), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 0.047218116634406226, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', None)
Agent drove right instead of left. (rewarded 0.05)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 1.39593385622
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.3959338562193986, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.40)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: -20.0322903089
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 4, 't': 21, 'action': 'left', 'reward': -20.03229030890732, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.03)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: left, reward: 0.979547835799
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'right', 'left'), 'deadline': 3, 't': 22, 'action': 'left', 'reward': 0.9795478357988234, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'right', 'left')
Agent drove left instead of right. (rewarded 0.98)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: left, reward: -39.8621220048
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 2, 't': 23, 'action': 'left', 'reward': -39.86212200478038, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.86)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.73471982559
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.7347198255872833, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.73)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 13
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (2, 6), deadline = 30
Simulating trial. . .
epsilon = 0.8781; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 1.08560022394
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'right', 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.085600223937, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'right', 'left')
Agent followed the waypoint right. (rewarded 1.09)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 1.83828256811
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.8382825681065973, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.84)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: -4.18154500067
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': -4.181545000666347, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.18)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: left, reward: -39.7308319788
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 27, 't': 3, 'action': 'left', 'reward': -39.73083197877787, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.73)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: left, reward: -9.58775383547
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': 'left', 'reward': -9.58775383547083, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.59)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 2.15379133691
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 2.1537913369089794, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.15)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 1.50026141313
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'right'), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 1.5002614131299046, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'right')
Agent drove right instead of forward. (rewarded 1.50)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: left, reward: 1.1668938856
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 1.1668938856049182, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent followed the waypoint left. (rewarded 1.17)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: left, reward: 1.3495911751
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': 1.3495911750979297, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 1.35)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: left, reward: -40.9352532747
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': -40.935253274748625, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.94)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: left, reward: -39.1269501262
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', None), 'deadline': 20, 't': 10, 'action': 'left', 'reward': -39.12695012621265, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.13)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: 0.380936958453
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 0.38093695845303344, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.38)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 0.370890460421
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': 0.3708904604213813, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.37)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: -5.64261593055
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', None, 'forward'), 'deadline': 17, 't': 13, 'action': None, 'reward': -5.64261593055209, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.64)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: -4.169590078
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', None, 'right'), 'deadline': 16, 't': 14, 'action': None, 'reward': -4.169590078001235, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.17)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 1.90494648926
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.9049464892561796, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.90)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: left, reward: 0.861310690324
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 0.8613106903237351, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent drove left instead of right. (rewarded 0.86)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 2.6445359593
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 13, 't': 17, 'action': None, 'reward': 2.644535959303376, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.64)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: -10.7536975463
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 12, 't': 18, 'action': 'forward', 'reward': -10.753697546329333, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.75)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: left, reward: -0.0646303899192
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 19, 'action': 'left', 'reward': -0.06463038991922188, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded -0.06)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 0.892999027113
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 0.892999027113268, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 0.89)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: 0.538174387353
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 21, 'action': 'left', 'reward': 0.5381743873531631, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.54)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: right, reward: 1.30180771148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 1.3018077114822384, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.30)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: -4.49924483234
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 7, 't': 23, 'action': None, 'reward': -4.499244832343628, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.50)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 0.693505256947
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 6, 't': 24, 'action': None, 'reward': 0.6935052569469953, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.69)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: right, reward: -20.7114716301
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 5, 't': 25, 'action': 'right', 'reward': -20.71147163013028, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.71)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: left, reward: -9.6946082171
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 4, 't': 26, 'action': 'left', 'reward': -9.694608217101358, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent attempted driving left through a red light. (rewarded -9.69)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: -10.1561654098
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 27, 'action': 'forward', 'reward': -10.156165409784055, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.16)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: forward, reward: 0.35237459684
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 2, 't': 28, 'action': 'forward', 'reward': 0.352374596840203, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.35)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: right, reward: -0.234050163997
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 1, 't': 29, 'action': 'right', 'reward': -0.23405016399748324, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove right instead of forward. (rewarded -0.23)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 14
\-------------------------
Environment.reset(): Trial set up with start = (2, 6), destination = (1, 3), deadline = 20
Simulating trial. . .
epsilon = 0.8694; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 1.75497527331
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.7549752733090755, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.75)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: right, reward: 1.54642927563
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.5464292756322937, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.55)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: -5.4376762061
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'right', 'right'), 'deadline': 18, 't': 2, 'action': None, 'reward': -5.437676206097494, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.44)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: right, reward: 0.652623272528
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 0.652623272527613, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 0.65)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: left, reward: -40.2251465555
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -40.225146555527196, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.23)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: forward, reward: 0.456459696896
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 0.45645969689572063, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 0.46)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: 2.50160673423
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 2.5016067342338033, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 2.50)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: -10.0036577854
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': -10.003657785352072, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.00)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: -39.0988445083
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -39.09884450832021, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.10)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: -10.7707739917
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': -10.770773991693867, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', 'left')
Agent attempted driving left through a red light. (rewarded -10.77)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: -10.5404503519
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -10.540450351933208, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent attempted driving left through a red light. (rewarded -10.54)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: left, reward: 0.286434952607
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 0.2864349526074603, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent drove left instead of forward. (rewarded 0.29)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: left, reward: 0.771848585666
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 0.7718485856660278, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 0.77)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 0.664559163556
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.6645591635562552, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.66)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: 0.422097166996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 0.42209716699552746, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove forward instead of right. (rewarded 0.42)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: -0.106330900941
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': -0.10633090094141084, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.11)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 1.00430620409
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'right'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.0043062040860848, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'right')
Agent drove forward instead of right. (rewarded 1.00)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: left, reward: -0.638432578501
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': -0.6384325785005408, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded -0.64)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: -5.9580965296
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': -5.9580965296017165, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.96)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: 0.262643174268
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.2626431742676897, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent drove right instead of left. (rewarded 0.26)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 15
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (6, 6), deadline = 20
Simulating trial. . .
epsilon = 0.8607; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: right, reward: 0.325630091973
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'right'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.3256300919734836, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'right')
Agent drove right instead of left. (rewarded 0.33)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: left, reward: 0.66955313808
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 0.6695531380802994, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 0.67)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: forward, reward: -9.70423555805
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': -9.704235558053366, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.70)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: left, reward: -9.3910933943
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -9.39109339429947, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.39)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: 0.689974417385
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 0.689974417384538, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.69)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: -5.91158212435
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': -5.911582124353247, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.91)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: 1.88704566795
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.8870456679525236, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.89)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 0.755611096732
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 0.7556110967324854, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.76)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 1.14793580783
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.1479358078293744, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.15)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: 1.48891539243
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.4889153924316822, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.49)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 1.85263570564
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.8526357056446892, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.85)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 2.58278145705
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 2.5827814570479126, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.58)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: -9.40834910025
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -9.408349100251854, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.41)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: -4.15262418994
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': -4.152624189936927, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.15)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 1.85333568378
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.853335683781769, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.85)
25% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 16
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (5, 2), deadline = 25
Simulating trial. . .
epsilon = 0.8521; alpha = 0.5000
Simulating trial. . .
epsilon = 0.8521; alpha = 0.5000
Simulating trial. . .
epsilon = 0.8521; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 1.42716996442
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.4271699644201064, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.43)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 0.221140091928
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.22114009192770356, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove right instead of left. (rewarded 0.22)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: -4.44795661474
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 23, 't': 2, 'action': None, 'reward': -4.447956614737036, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.45)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: 1.72380263604
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.7238026360438543, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove right instead of left. (rewarded 1.72)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 1.50674743078
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.506747430782465, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.51)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 2.05840550977
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.0584055097659677, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.06)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 1.64055593384
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.640555933837631, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.64)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 1.33960422915
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.339604229145686, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 1.34)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: forward, reward: -9.12629006528
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -9.12629006528105, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent attempted driving forward through a red light. (rewarded -9.13)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: -4.80813482322
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': -4.808134823222884, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.81)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: -5.85853194599
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': -5.858531945989437, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.86)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: left, reward: -10.8174284978
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': -10.81742849775894, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent attempted driving left through a red light. (rewarded -10.82)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: left, reward: -40.3318639111
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', 'left'), 'deadline': 13, 't': 12, 'action': 'left', 'reward': -40.331863911117814, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.33)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 0.914315373611
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': None, 'reward': 0.9143153736105099, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.91)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: forward, reward: 0.319165301689
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 0.3191653016889494, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 0.32)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: left, reward: 2.28895505449
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 2.2889550544886994, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.29)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: left, reward: 0.420553641527
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 0.4205536415269281, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.42)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: forward, reward: 0.633557902211
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 0.6335579022108067, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent drove forward instead of right. (rewarded 0.63)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: -5.65851822326
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': -5.658518223259433, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.66)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: forward, reward: -9.81785148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': -9.817851479996406, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.82)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: left, reward: -39.9927750569
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 5, 't': 20, 'action': 'left', 'reward': -39.992775056913885, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.99)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: left, reward: 0.948348454024
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 0.9483484540238315, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove left instead of right. (rewarded 0.95)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.03472753277
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 1.0347275327697705, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.03)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: left, reward: -39.737156215
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 2, 't': 23, 'action': 'left', 'reward': -39.73715621496792, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.74)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: left, reward: -9.40174555774
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 1, 't': 24, 'action': 'left', 'reward': -9.401745557738256, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.40)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 17
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (1, 3), deadline = 20
Simulating trial. . .
epsilon = 0.8437; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: 1.8530040825
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.8530040824984235, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.85)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: right, reward: 0.312800881428
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.31280088142782836, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.31)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: -5.14762073909
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 18, 't': 2, 'action': None, 'reward': -5.147620739093293, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.15)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.77258217014
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.772582170135164, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.77)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: -10.0249577259
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -10.024957725897739, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.02)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: -5.64063479738
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 15, 't': 5, 'action': None, 'reward': -5.640634797384775, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.64)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: -0.05236276261
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -0.05236276260998651, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded -0.05)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: 1.21495946833
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.2149594683268572, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 1.21)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 1.40626471509
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.4062647150887715, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 1.41)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 1.38005768628
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.380057686275384, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent drove right instead of left. (rewarded 1.38)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 2.60331150597
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.6033115059665546, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.60)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: -4.72092438015
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': -4.720924380149356, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.72)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: -4.9675821745
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': -4.967582174504348, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.97)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: left, reward: -10.0951123971
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': -10.095112397053857, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.10)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 2.49104611117
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.491046111169818, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.49)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: right, reward: 1.54728899604
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.5472889960385605, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.55)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: -40.1174360993
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': -40.11743609926232, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.12)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: -10.7381855523
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -10.738185552322399, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.74)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 0.602718291649
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'right'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 0.6027182916486244, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'right')
Agent drove right instead of forward. (rewarded 0.60)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: forward, reward: -9.34372865484
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -9.343728654839511, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.34)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 18
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (1, 6), deadline = 20
Simulating trial. . .
epsilon = 0.8353; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 1.14392011002
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.1439201100204661, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 1.14)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 0.96473901114
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 0.9647390111396797, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove forward instead of left. (rewarded 0.96)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 1.13958253618
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.1395825361758902, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.14)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: forward, reward: 1.33454932034
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'left', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.3345493203368246, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'left', None)
Agent drove forward instead of right. (rewarded 1.33)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 1.54215804324
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.5421580432356738, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.54)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: forward, reward: -9.39346070716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': -9.393460707156727, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.39)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: 2.35853857935
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'left'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.3585385793505815, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'left')
Agent followed the waypoint right. (rewarded 2.36)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: 2.60724449734
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.6072444973394733, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.61)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: -19.5131120303
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'right'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': -19.513112030262366, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'right')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.51)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 0.82721245059
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.8272124505897576, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.83)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 0.535862580258
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 0.5358625802578293, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.54)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: -10.2082714743
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': -10.208271474254087, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.21)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.33156701657
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.3315670165731337, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.33)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 0.154471092642
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 7, 't': 13, 'action': None, 'reward': 0.15447109264176817, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.15)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: -5.86339496085
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'right'), 'deadline': 6, 't': 14, 'action': None, 'reward': -5.863394960849209, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.86)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: -9.82358172873
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -9.823581728728328, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -9.82)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.32445555217
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 2.324455552166249, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.32)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: right, reward: 0.481389380443
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.48138938044333146, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove right instead of forward. (rewarded 0.48)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: -0.766600218963
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': -0.7666002189634353, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent drove forward instead of left. (rewarded -0.77)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 1.53967074043
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.5396707404265688, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.54)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 19
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (4, 3), deadline = 20
Simulating trial. . .
epsilon = 0.8270; alpha = 0.5000
Simulating trial. . .
epsilon = 0.8270; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: -4.43489908319
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': -4.4348990831860595, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.43)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: -5.72645607006
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': -5.726456070061814, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.73)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 0.881522845819
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.8815228458187042, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.88)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 1.93303026743
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.9330302674250892, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 1.93)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: -9.26877220287
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': -9.268772202872608, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.27)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 1.53808438663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.5380843866285274, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.54)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 0.821360055788
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 0.8213600557878966, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.82)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: -4.01013528733
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': -4.010135287331558, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.01)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 0.817297596654
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.8172975966535115, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.82)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 1.61350084191
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.6135008419069767, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.61)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.26623502051
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.2662350205066564, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 1.27)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 0.567277837646
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.567277837645769, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.57)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 1.06494113869
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.0649411386923966, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.06)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 0.468946834569
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', 'left'), 'deadline': 7, 't': 13, 'action': None, 'reward': 0.4689468345686857, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.47)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: left, reward: 1.84817937953
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 1.8481793795327592, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.85)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 0.902854451263
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 0.9028544512631218, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.90)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 1.20090067426
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 1.2009006742582926, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 1.20)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 0.421762848916
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.4217628489159684, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.42)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: -4.90320387089
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 2, 't': 18, 'action': None, 'reward': -4.903203870894691, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.90)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.78160499934
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'right'), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.7816049993423622, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, 'forward', 'right')
Agent properly idled at a red light. (rewarded 1.78)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 20
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (7, 5), deadline = 30
Simulating trial. . .
epsilon = 0.8187; alpha = 0.5000
Simulating trial. . .
epsilon = 0.8187; alpha = 0.5000
Simulating trial. . .
epsilon = 0.8187; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 2.46458139823
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.46458139822911, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.46)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 2.57383092439
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'right'), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.573830924392218, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'right')
Agent properly idled at a red light. (rewarded 2.57)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: left, reward: -40.9135035755
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 28, 't': 2, 'action': 'left', 'reward': -40.91350357550922, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.91)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: 1.38516295537
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 1.3851629553677953, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent drove right instead of left. (rewarded 1.39)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 1.3241385962
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.3241385962024432, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.32)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: 2.82544304621
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 2.825443046213218, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.83)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 0.139316790482
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 0.13931679048241496, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.14)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 2.21638805627
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 2.2163880562722356, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.22)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: -10.8897824812
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': -10.889782481183726, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent attempted driving forward through a red light. (rewarded -10.89)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 0.905321754957
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 0.9053217549569696, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded 0.91)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: left, reward: 2.66431909261
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 2.664319092611076, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.66)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: left, reward: -10.4117096426
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': -10.411709642633896, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.41)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.35669321537
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.3566932153675442, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.36)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: 1.58280738576
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.5828073857603666, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.58)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: -10.4038027444
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 14, 'action': 'left', 'reward': -10.403802744381654, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.40)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: None, reward: 2.70737690124
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 15, 'action': None, 'reward': 2.7073769012367315, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.71)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: -39.5401598949
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': -39.54015989488769, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.54)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: 2.0452528453
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 13, 't': 17, 'action': 'left', 'reward': 2.0452528452951855, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 2.05)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: left, reward: -0.0566096116238
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 18, 'action': 'left', 'reward': -0.056609611623814615, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded -0.06)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: -5.66599378219
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'forward'), 'deadline': 11, 't': 19, 'action': None, 'reward': -5.665993782192154, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.67)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 1.94759161354
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 1.9475916135449232, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 1.95)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 0.943979535952
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 0.9439795359522053, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 0.94)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 0.19622736763
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 0.19622736762977322, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 0.20)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 2.18207067515
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 7, 't': 23, 'action': None, 'reward': 2.182070675145199, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 2.18)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 0.827925851584
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 0.8279258515841983, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.83)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 2.30126816854
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 5, 't': 25, 'action': None, 'reward': 2.3012681685424177, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.30)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 0.876221953575
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 4, 't': 26, 'action': None, 'reward': 0.8762219535745297, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 0.88)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 0.724379390154
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 3, 't': 27, 'action': None, 'reward': 0.7243793901543065, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.72)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: -4.3538245581
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 2, 't': 28, 'action': None, 'reward': -4.3538245581000385, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.35)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: forward, reward: -10.5571835553
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 1, 't': 29, 'action': 'forward', 'reward': -10.557183555304327, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.56)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 21
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (2, 4), deadline = 30
Simulating trial. . .
epsilon = 0.8106; alpha = 0.5000
Simulating trial. . .
epsilon = 0.8106; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: 1.76980431464
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'left'), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.7698043146400222, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.77)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: 1.72705606228
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'left'), 'deadline': 29, 't': 1, 'action': 'left', 'reward': 1.7270560622769533, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'left')
Agent followed the waypoint left. (rewarded 1.73)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: -4.12944588233
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 28, 't': 2, 'action': None, 'reward': -4.129445882333694, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.13)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: -4.05870961402
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 27, 't': 3, 'action': None, 'reward': -4.058709614020739, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.06)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: 0.858476105895
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 0.8584761058949447, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent drove left instead of forward. (rewarded 0.86)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: forward, reward: 1.44879518454
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.4487951845355345, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.45)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 1.81155846445
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.8115584644462541, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.81)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 2.59116696605
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'left'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 2.5911669660503778, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 2.59)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 0.877407355715
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 0.8774073557151673, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.88)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: left, reward: -39.2161688748
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': -39.21616887482132, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.22)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: left, reward: -10.2484138171
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': 'left', 'reward': -10.248413817094994, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.25)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 1.39242287741
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.3924228774109537, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.39)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 0.116409370366
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 0.11640937036645338, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent drove right instead of left. (rewarded 0.12)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 2.78358839936
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 2.7835883993582695, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.78)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: left, reward: -10.2288458766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 16, 't': 14, 'action': 'left', 'reward': -10.228845876628988, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent attempted driving left through a red light. (rewarded -10.23)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: left, reward: -0.112914505611
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 15, 't': 15, 'action': 'left', 'reward': -0.11291450561057148, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent drove left instead of right. (rewarded -0.11)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 2.64968063705
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 2.6496806370501673, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.65)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: forward, reward: -9.89329244749
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': -9.893292447485443, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.89)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: left, reward: -10.8874350797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 12, 't': 18, 'action': 'left', 'reward': -10.887435079679593, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.89)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: left, reward: 0.33407992913
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 11, 't': 19, 'action': 'left', 'reward': 0.33407992912979034, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent drove left instead of right. (rewarded 0.33)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: -4.02597206997
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 20, 'action': None, 'reward': -4.0259720699668256, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.03)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: 0.825815678381
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 0.8258156783809061, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.83)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: right, reward: 2.4990134467
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 2.4990134466953204, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.50)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: left, reward: 0.155633403843
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 7, 't': 23, 'action': 'left', 'reward': 0.1556334038425996, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.16)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 1.0240235904
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 1.0240235904023187, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.02)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: -39.2106530104
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 5, 't': 25, 'action': 'left', 'reward': -39.210653010412116, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.21)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: -10.8744828864
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 4, 't': 26, 'action': 'forward', 'reward': -10.874482886410856, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.87)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: right, reward: 0.898917656722
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 3, 't': 27, 'action': 'right', 'reward': 0.8989176567223145, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 0.90)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: forward, reward: -0.589297427107
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 28, 'action': 'forward', 'reward': -0.5892974271067567, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded -0.59)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: right, reward: 0.633960338681
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 1, 't': 29, 'action': 'right', 'reward': 0.6339603386811248, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.63)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 22
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (1, 2), deadline = 25
Simulating trial. . .
epsilon = 0.8025; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: -4.06953696898
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': -4.069536968981149, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.07)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: left, reward: 2.87926126232
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 2.879261262315996, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.88)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: left, reward: 0.518345650482
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 0.5183456504822381, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove left instead of forward. (rewarded 0.52)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: right, reward: 1.52380232466
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.5238023246604357, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.52)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: left, reward: -10.0221610627
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'left', 'reward': -10.022161062688284, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.02)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 0.195165603639
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 0.1951656036388868, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.20)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: left, reward: 0.552873411606
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 0.5528734116056507, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove left instead of forward. (rewarded 0.55)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: forward, reward: 1.79918324998
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.7991832499788138, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 1.80)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 2.47403861497
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'forward', None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 2.4740386149650506, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'forward', None)
Agent followed the waypoint right. (rewarded 2.47)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 2.44553285196
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'right'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.4455328519578, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'right')
Agent properly idled at a red light. (rewarded 2.45)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 0.843038189811
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'right'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.8430381898105825, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'right')
Agent followed the waypoint right. (rewarded 0.84)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 23
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (3, 2), deadline = 25
Simulating trial. . .
epsilon = 0.7945; alpha = 0.5000
Simulating trial. . .
epsilon = 0.7945; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: left, reward: -39.7462826178
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'right'), 'deadline': 25, 't': 0, 'action': 'left', 'reward': -39.746282617801434, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.75)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: left, reward: -40.3756742586
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -40.37567425863127, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.38)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: forward, reward: -10.8979561649
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': -10.897956164871312, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.90)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: forward, reward: 1.80993046752
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.809930467516763, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.81)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: right, reward: 1.70793407838
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.7079340783778094, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 1.71)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: 1.94332013657
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.9433201365721877, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.94)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: left, reward: -10.5495424164
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': -10.549542416364508, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent attempted driving left through a red light. (rewarded -10.55)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 0.982985871209
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 0.9829858712092021, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.98)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: -0.0194890158558
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': -0.01948901585583307, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.02)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: left, reward: -10.3130588772
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'left'), 'deadline': 16, 't': 9, 'action': 'left', 'reward': -10.313058877237776, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'left')
Agent attempted driving left through a red light. (rewarded -10.31)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 2.07136581765
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.071365817652226, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.07)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: right, reward: 0.671865495684
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 0.6718654956840738, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.67)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: left, reward: 1.09251945215
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'right', None), 'deadline': 13, 't': 12, 'action': 'left', 'reward': 1.0925194521544446, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'right', None)
Agent followed the waypoint left. (rewarded 1.09)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: left, reward: -9.86874970879
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'left', 'reward': -9.868749708794542, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.87)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: forward, reward: -39.9955700514
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': -39.99557005138012, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.00)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: forward, reward: -9.30917578082
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': -9.309175780822338, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.31)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: -0.343951875059
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 16, 'action': None, 'reward': -0.34395187505948377, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.34)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: right, reward: 1.42640202171
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.426402021709234, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove right instead of forward. (rewarded 1.43)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: right, reward: 1.12033571066
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'right'), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 1.1203357106552623, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'right')
Agent drove right instead of left. (rewarded 1.12)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: -0.452158841675
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': -0.45215884167485576, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent drove forward instead of left. (rewarded -0.45)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 0.738715378281
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 0.73871537828137, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 0.74)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 1.275521482
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 1.2755214819954699, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.28)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.07225323736
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 1.0722532373613494, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.07)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 0.265683184927
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 23, 'action': None, 'reward': 0.2656831849272756, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.27)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 0.534440625748
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 24, 'action': 'right', 'reward': 0.5344406257481831, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.53)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 24
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (3, 2), deadline = 30
Simulating trial. . .
epsilon = 0.7866; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: left, reward: 1.45096060506
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 1.450960605059771, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove left instead of right. (rewarded 1.45)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: -5.4443962212
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': -5.444396221198149, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.44)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 2.55622855934
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 2.55622855933973, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.56)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 2.07350441138
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 2.073504411380841, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.07)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 0.90111210032
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 0.9011121003201558, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.90)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 0.122467031626
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 0.12246703162555861, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 0.12)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: -9.13813850993
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': -9.138138509930018, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.14)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 1.70395263784
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 1.7039526378365526, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.70)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 1.09500443998
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 1.0950044399778194, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent drove right instead of left. (rewarded 1.10)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: -20.5672901716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 21, 't': 9, 'action': 'right', 'reward': -20.567290171616374, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.57)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.72302486278
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.7230248627828666, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.72)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 0.167925280579
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 0.167925280579453, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove right instead of forward. (rewarded 0.17)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: 1.43954060564
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': 1.439540605639848, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.44)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: 1.06863913812
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 1.068639138121104, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.07)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 2.68028912184
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': 2.680289121837492, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.68)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 1.40186448252
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.4018644825238251, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.40)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: left, reward: -9.86440212425
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 14, 't': 16, 'action': 'left', 'reward': -9.864402124246075, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.86)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: left, reward: 2.14156526584
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 13, 't': 17, 'action': 'left', 'reward': 2.1415652658353928, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.14)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 25
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (5, 7), deadline = 20
Simulating trial. . .
epsilon = 0.7788; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: 2.66181939685
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.6618193968476915, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', 'left')
Agent properly idled at a red light. (rewarded 2.66)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: -9.45238818669
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -9.452388186693481, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.45)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 2.34793566705
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.3479356670537834, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 2.35)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: None, reward: 1.88473334466
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.8847333446640648, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.88)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: left, reward: 0.529844983224
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 0.5298449832243949, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 0.53)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 2.23466152082
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.234661520815129, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.23)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 2.51640895035
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.5164089503522247, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.52)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: -5.01966467655
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': -5.019664676548038, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.02)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: right, reward: 1.91824559839
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.9182455983948632, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.92)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: left, reward: -0.129683536026
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': -0.12968353602642058, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove left instead of forward. (rewarded -0.13)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: 0.149906183153
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 0.1499061831529599, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent drove forward instead of right. (rewarded 0.15)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: 1.48798562237
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 1.4879856223703412, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 1.49)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: forward, reward: 0.781986476852
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 0.7819864768518814, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 0.78)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: left, reward: -40.095428952
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': -40.09542895203505, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.10)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: right, reward: 0.706763366723
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 0.7067633667232812, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent drove right instead of left. (rewarded 0.71)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 0.895985174105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 0.8959851741048386, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.90)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: left, reward: -0.00426510430884
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': -0.004265104308843304, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove left instead of forward. (rewarded -0.00)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: forward, reward: 0.437138351839
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'right', 'forward'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.43713835183937355, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'right', 'forward')
Agent drove forward instead of right. (rewarded 0.44)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: left, reward: -39.9476230548
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'right', 'forward'), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -39.94762305483548, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'right', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.95)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: None, reward: 1.56369693588
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.563696935882462, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.56)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 26
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (7, 4), deadline = 25
Simulating trial. . .
epsilon = 0.7711; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 1.78154681808
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.7815468180781675, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 1.78)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 2.19945197333
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.1994519733283924, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.20)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: left, reward: -9.94700004654
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': -9.947000046539655, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent attempted driving left through a red light. (rewarded -9.95)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 1.20813984293
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.2081398429306314, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.21)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: -4.03744604144
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': -4.037446041437608, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.04)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: forward, reward: -39.1508220841
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': -39.15082208409768, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.15)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 1.93370336067
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'right'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.9337033606732197, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 1.93)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: left, reward: -39.0953207896
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'left', None, 'right'), 'deadline': 18, 't': 7, 'action': 'left', 'reward': -39.09532078956158, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.10)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: left, reward: -9.74298456555
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': -9.742984565546015, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.74)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: left, reward: 0.612753008185
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 0.61275300818539, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove left instead of right. (rewarded 0.61)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: left, reward: -39.3430166894
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'right'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': -39.34301668943092, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.34)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: 0.379033189926
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 0.37903318992580515, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 0.38)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: right, reward: 1.61000967733
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.6100096773271328, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.61)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: left, reward: -40.2390204774
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': -40.23902047739932, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.24)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: right, reward: -19.6395981382
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': -19.639598138243866, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.64)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: forward, reward: -9.9566271649
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': -9.956627164900727, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.96)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: right, reward: 1.09626944715
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 1.0962694471462875, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.10)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: -9.98854223997
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': -9.98854223996822, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.99)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: left, reward: -39.0104794099
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 7, 't': 18, 'action': 'left', 'reward': -39.01047940985741, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.01)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 2.38498573803
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 6, 't': 19, 'action': None, 'reward': 2.3849857380262973, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.38)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 1.33190777499
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.3319077749865718, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.33)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 1.24757906533
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 1.2475790653330767, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.25)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: -5.67168065028
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 3, 't': 22, 'action': None, 'reward': -5.671680650282796, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.67)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 1.22153785937
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.2215378593658348, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.22)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 0.286827015417
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.2868270154171, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.29)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 27
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (2, 6), deadline = 30
Simulating trial. . .
epsilon = 0.7634; alpha = 0.5000
Simulating trial. . .
epsilon = 0.7634; alpha = 0.5000
Simulating trial. . .
epsilon = 0.7634; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 1.79846820489
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.7984682048859617, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'left')
Agent drove right instead of left. (rewarded 1.80)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: left, reward: -39.1394355293
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 29, 't': 1, 'action': 'left', 'reward': -39.13943552932898, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.14)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: 1.03204169825
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 1.0320416982467222, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 1.03)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 1.44732977692
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.4473297769222455, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.45)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 0.345573184389
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 0.34557318438857865, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 0.35)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: right, reward: 1.09561156701
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.095611567005959, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 1.10)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: left, reward: -19.4605773363
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 24, 't': 6, 'action': 'left', 'reward': -19.460577336255458, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.46)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: forward, reward: -9.68726645224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': -9.687266452244913, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.69)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: forward, reward: -10.3455064345
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'right', 'forward'), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': -10.345506434483209, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', 'forward')
Agent attempted driving forward through a red light. (rewarded -10.35)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 1.22722009531
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 1.2272200953060997, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.23)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: 1.80302975542
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 1.8030297554180814, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.80)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: -5.49256874378
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 19, 't': 11, 'action': None, 'reward': -5.49256874377738, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.49)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: 2.82888258632
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': 2.8288825863188727, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.83)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 1.16355630961
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.1635563096073234, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.16)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: -5.5308509377
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': -5.530850937698554, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.53)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 0.550552624915
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 0.5505526249145553, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove right instead of left. (rewarded 0.55)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: -5.79208902243
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 14, 't': 16, 'action': None, 'reward': -5.792089022428351, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.79)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: left, reward: -39.3481777403
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 13, 't': 17, 'action': 'left', 'reward': -39.348177740338095, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.35)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 1.57314149673
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'forward'), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.5731414967333617, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.57)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: -9.60061191188
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': -9.600611911880607, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -9.60)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: left, reward: -10.2714520931
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 10, 't': 20, 'action': 'left', 'reward': -10.271452093146943, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.27)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 0.602450937384
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 9, 't': 21, 'action': None, 'reward': 0.6024509373840602, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.60)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: left, reward: -19.5858335253
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 8, 't': 22, 'action': 'left', 'reward': -19.58583352531169, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.59)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: left, reward: 0.35370344142
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 7, 't': 23, 'action': 'left', 'reward': 0.3537034414198249, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.35)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.01400661678
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 6, 't': 24, 'action': None, 'reward': 2.0140066167799233, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.01)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: left, reward: 0.870182346759
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 25, 'action': 'left', 'reward': 0.8701823467587448, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 0.87)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 1.12339213207
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'forward'), 'deadline': 4, 't': 26, 'action': None, 'reward': 1.123392132069907, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.12)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: -0.0754382035422
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 3, 't': 27, 'action': 'right', 'reward': -0.07543820354224784, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded -0.08)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 0.931068317345
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 2, 't': 28, 'action': 'right', 'reward': 0.931068317344928, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove right instead of left. (rewarded 0.93)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: left, reward: -39.4403087424
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'left', 'left', 'right'), 'deadline': 1, 't': 29, 'action': 'left', 'reward': -39.44030874243443, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'left', 'left', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.44)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 28
\-------------------------
Environment.reset(): Trial set up with start = (5, 2), destination = (4, 5), deadline = 20
Simulating trial. . .
epsilon = 0.7558; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: 0.268747077693
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 0.26874707769264217, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.27)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: forward, reward: 0.27640520313
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 0.27640520312994166, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent drove forward instead of right. (rewarded 0.28)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: forward, reward: 1.27342489197
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.273424891965329, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove forward instead of right. (rewarded 1.27)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 1.45200553576
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.4520055357642887, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.45)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: -5.98277828636
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': -5.982778286362176, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.98)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: 0.0962399182686
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 0.09623991826858647, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 0.10)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.2422510794
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.2422510793993662, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.24)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: -19.3041023028
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, 'right', 'right'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': -19.304102302840903, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.30)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: left, reward: 1.07456204966
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.0745620496604391, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove left instead of right. (rewarded 1.07)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: 1.77059240507
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.7705924050673876, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.77)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: left, reward: 0.0551054783056
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 0.055105478305638056, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 0.06)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: 0.981125344449
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 0.9811253444488666, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 0.98)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 0.0056955516759
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 0.005695551675899635, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent drove forward instead of right. (rewarded 0.01)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: forward, reward: 1.38402331007
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.3840233100654271, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 1.38)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 0.291083231613
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 0.29108323161268845, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.29)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 0.821325929892
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 0.8213259298921034, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent drove right instead of left. (rewarded 0.82)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: -5.43646597891
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 4, 't': 16, 'action': None, 'reward': -5.436465978906687, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.44)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: -20.0519595704
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': -20.051959570417036, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.05)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.34674324828
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.3467432482812043, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.35)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 0.743688768822
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.7436887688220242, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.74)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 29
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (3, 7), deadline = 25
Simulating trial. . .
epsilon = 0.7483; alpha = 0.5000
Simulating trial. . .
epsilon = 0.7483; alpha = 0.5000
Simulating trial. . .
epsilon = 0.7483; alpha = 0.5000
Simulating trial. . .
epsilon = 0.7483; alpha = 0.5000
Simulating trial. . .
epsilon = 0.7483; alpha = 0.5000
Simulating trial. . .
epsilon = 0.7483; alpha = 0.5000
Simulating trial. . .
epsilon = 0.7483; alpha = 0.5000
Simulating trial. . .
epsilon = 0.7483; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: left, reward: 1.01838768014
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 1.0183876801409284, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.02)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: left, reward: 0.761384181571
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 0.7613841815706364, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.76)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: forward, reward: 1.00993319615
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 1.0099331961505444, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent drove forward instead of right. (rewarded 1.01)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: 2.22271943774
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 2.2227194377415076, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 2.22)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: -10.5360980409
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -10.536098040889732, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.54)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 1.37880525996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.3788052599576432, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove right instead of left. (rewarded 1.38)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 1.10053597794
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.1005359779368689, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.10)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: -5.13797292604
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 18, 't': 7, 'action': None, 'reward': -5.137972926042921, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.14)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 2.19861249189
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 2.198612491893014, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent followed the waypoint right. (rewarded 2.20)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: 1.18827137089
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.1882713708862391, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.19)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: 1.27440803749
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.2744080374937945, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.27)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: -10.5097907182
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': -10.509790718230798, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -10.51)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: forward, reward: -10.3722086189
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': -10.372208618920384, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.37)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: 0.0584945976444
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 0.05849459764438292, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.06)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: -4.37737901688
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': -4.3773790168788995, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.38)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 1.58597564539
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.5859756453885292, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.59)
36% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 30
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (4, 7), deadline = 20
Simulating trial. . .
epsilon = 0.7408; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 1.42792445573
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.4279244557269963, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', 'left')
Agent followed the waypoint right. (rewarded 1.43)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 2.01066556421
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.0106655642080344, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.01)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: left, reward: -9.22668069215
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 18, 't': 2, 'action': 'left', 'reward': -9.226680692153925, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent attempted driving left through a red light. (rewarded -9.23)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 1.60830298874
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.6083029887424323, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.61)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: -9.75574103497
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': -9.755741034974797, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.76)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: left, reward: 1.40379747241
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.4037974724079818, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove left instead of forward. (rewarded 1.40)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: 2.54328578449
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.543285784485484, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.54)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: -9.93176257892
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -9.931762578915611, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.93)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: -39.6093684189
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'right'), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -39.60936841891608, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.61)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: -5.88275772695
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': -5.882757726946062, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.88)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: forward, reward: 1.78498023491
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.784980234907055, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.78)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: left, reward: -0.250692030882
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': -0.2506920308816962, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded -0.25)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: left, reward: 0.398805569744
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 0.3988055697439481, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent drove left instead of right. (rewarded 0.40)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: -0.14292026218
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': -0.14292026217968268, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded -0.14)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: left, reward: 0.823507341902
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 0.8235073419022312, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 0.82)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: 0.887875835195
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 0.8878758351954046, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 0.89)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.57875220763
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.5787522076312495, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.58)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: -0.0744690460193
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': -0.0744690460193086, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent drove left instead of forward. (rewarded -0.07)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: forward, reward: -0.467452689186
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'forward'), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': -0.4674526891859394, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'forward')
Agent drove forward instead of right. (rewarded -0.47)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: forward, reward: 0.672464240329
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 0.6724642403294906, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'forward', None, None)
Agent drove forward instead of right. (rewarded 0.67)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 31
\-------------------------
Environment.reset(): Trial set up with start = (3, 2), destination = (8, 7), deadline = 20
Simulating trial. . .
epsilon = 0.7334; alpha = 0.5000
Simulating trial. . .
epsilon = 0.7334; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 0.560436801229
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 0.5604368012292118, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.56)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: -4.62454464905
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 19, 't': 1, 'action': None, 'reward': -4.6245446490484525, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.62)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: 0.969494030951
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 0.9694940309513339, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove forward instead of left. (rewarded 0.97)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: 1.72708465601
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.7270846560140716, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 1.73)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: left, reward: -40.7457613602
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -40.74576136024491, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.75)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: -4.18848650994
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': -4.18848650994418, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.19)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 1.07092128093
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.070921280927224, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.07)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 0.73405160828
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.7340516082795585, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.73)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.92169204437
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.9216920443716277, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.92)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.58267336948
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.5826733694837714, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.58)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: forward, reward: 0.0545206154228
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 0.05452061542277209, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.05)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: left, reward: 2.20953558359
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 2.2095355835945547, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 2.21)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: left, reward: 1.0302340737
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 1.0302340736954771, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.03)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: right, reward: 1.16536685547
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.165366855474769, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 1.17)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: right, reward: -20.5212812131
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': -20.521281213125825, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.52)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 0.458334153962
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 0.4583341539616377, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.46)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: -10.1493209063
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': -10.149320906306949, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -10.15)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: -9.5881838205
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -9.588183820500419, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.59)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: 1.05217287084
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 1.0521728708358744, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent drove right instead of left. (rewarded 1.05)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 1.85463268544
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'right'), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.8546326854403283, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 1.85)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 32
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (1, 3), deadline = 30
Simulating trial. . .
epsilon = 0.7261; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 1.77844444155
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.7784444415524163, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.78)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 2.63988286392
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 2.639882863919617, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.64)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 2.28834926105
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'right'), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 2.2883492610513567, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'right')
Agent followed the waypoint forward. (rewarded 2.29)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.64582418198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.645824181976808, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.65)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.48469690977
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.484696909772771, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.48)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: -4.42561599354
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': -4.425615993542445, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.43)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 1.88103165315
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 1.881031653148971, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 1.88)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 1.04036009892
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.040360098919018, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 1.04)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: -10.5493905724
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': -10.549390572417597, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.55)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: -9.3738777897
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': -9.373877789697708, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.37)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: -20.960768721
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 20, 't': 10, 'action': 'left', 'reward': -20.96076872104044, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.96)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: 0.929022997284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 0.929022997283526, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.93)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: 0.214748330374
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 0.21474833037379126, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.21)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: -19.061849933
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'left', 'right', 'forward'), 'deadline': 17, 't': 13, 'action': 'left', 'reward': -19.061849933024426, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.06)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: -19.6129675814
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'right'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'right', 'right', 'forward'), 'deadline': 16, 't': 14, 'action': 'left', 'reward': -19.61296758143903, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'right', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.61)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: -5.27192939604
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 15, 't': 15, 'action': None, 'reward': -5.2719293960393125, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.27)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 0.894328855649
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 0.8943288556488902, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 0.89)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: None, reward: 1.12349186662
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.123491866623966, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.12)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 2.62317959049
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 2.623179590491165, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.62)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: left, reward: -39.802308542
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'right'), 'deadline': 11, 't': 19, 'action': 'left', 'reward': -39.802308542026175, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.80)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 2.22983373347
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 20, 'action': None, 'reward': 2.229833733467608, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.23)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: -10.1644308406
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': -10.16443084057185, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.16)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: -5.29819739633
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 8, 't': 22, 'action': None, 'reward': -5.298197396325532, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.30)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 0.773517298819
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 0.7735172988187181, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent drove right instead of forward. (rewarded 0.77)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: forward, reward: -9.41152279182
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 6, 't': 24, 'action': 'forward', 'reward': -9.411522791820689, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.41)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: -19.1965068544
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'right', None, 'right'), 'deadline': 5, 't': 25, 'action': 'left', 'reward': -19.19650685443869, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.20)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: 1.94520154414
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 26, 'action': 'left', 'reward': 1.9452015441441675, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.95)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 2.21724814662
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 27, 'action': None, 'reward': 2.217248146619946, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.22)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 0.36292100794
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 2, 't': 28, 'action': 'right', 'reward': 0.3629210079396826, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 0.36)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: forward, reward: 0.161109085444
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 1, 't': 29, 'action': 'forward', 'reward': 0.16110908544369762, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.16)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 33
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (7, 3), deadline = 25
Simulating trial. . .
epsilon = 0.7189; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: forward, reward: 0.331736934023
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', 'forward'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 0.3317369340233771, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', 'forward')
Agent drove forward instead of right. (rewarded 0.33)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: right, reward: 1.86245507252
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.8624550725181515, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 1.86)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: left, reward: 1.04731627521
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 1.0473162752109044, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', None)
Agent drove left instead of forward. (rewarded 1.05)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: forward, reward: -39.4860328828
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': -39.486032882849265, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.49)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: left, reward: -10.0755728589
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': -10.075572858914933, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.08)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: right, reward: 0.977310624972
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.9773106249722254, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 0.98)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 0.579716249012
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.5797162490120497, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.58)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: right, reward: 1.61285062449
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.6128506244895158, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.61)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: right, reward: 1.31315182992
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.3131518299214533, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.31)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: right, reward: 0.901811312506
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.9018113125057168, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.90)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 0.672645172929
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.6726451729290318, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded 0.67)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: -4.38015068032
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 14, 't': 11, 'action': None, 'reward': -4.380150680315198, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.38)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: -5.4677871864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 13, 't': 12, 'action': None, 'reward': -5.467787186400445, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.47)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: right, reward: 2.20998398348
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 2.2099839834817523, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.21)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: right, reward: 1.42227266224
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 1.4222726622414124, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.42)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: forward, reward: -10.2370861296
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': -10.237086129610681, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.24)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: left, reward: -39.928374066
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 9, 't': 16, 'action': 'left', 'reward': -39.92837406595087, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.93)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: right, reward: 1.40766514887
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.4076651488697223, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.41)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 0.573072680675
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 0.5730726806746309, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.57)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: -9.52449571356
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': -9.524495713556414, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.52)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 1.72721288861
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.727212888606846, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.73)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: left, reward: 1.2682534951
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 1.2682534950987594, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent followed the waypoint left. (rewarded 1.27)
12% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 34
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (7, 6), deadline = 20
Simulating trial. . .
epsilon = 0.7118; alpha = 0.5000
Simulating trial. . .
epsilon = 0.7118; alpha = 0.5000
Simulating trial. . .
epsilon = 0.7118; alpha = 0.5000
Simulating trial. . .
epsilon = 0.7118; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: left, reward: 1.73449137258
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.7344913725845315, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 1.73)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 2.17518124396
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.1751812439596145, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.18)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: -10.4674667856
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': -10.467466785550998, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.47)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 1.59615194547
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.5961519454718518, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.60)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: 1.85255762648
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.8525576264752592, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 1.85)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 1.58364990796
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.5836499079591517, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 1.58)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: -5.90387933391
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 14, 't': 6, 'action': None, 'reward': -5.903879333913401, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.90)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: -5.63451717438
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': -5.634517174377088, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.63)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: left, reward: 1.08897830377
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.0889783037666803, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent drove left instead of right. (rewarded 1.09)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: -10.1298104577
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': -10.129810457736145, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.13)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 1.0078947043
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'right'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.0078947042959956, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.01)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: -10.5341047424
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': -10.534104742359114, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.53)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: 0.576020260814
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'right'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.5760202608141711, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'right')
Agent drove right instead of left. (rewarded 0.58)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: -0.25223696949
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': -0.25223696948972474, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded -0.25)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: -20.9086014694
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': -20.908601469392597, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.91)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 0.606903738082
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 0.6069037380822742, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 0.61)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: -4.33479861483
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 4, 't': 16, 'action': None, 'reward': -4.334798614825479, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.33)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: -40.3410758189
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -40.34107581891764, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.34)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 0.889960919885
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.8899609198846399, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.89)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 0.244045765172
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.24404576517183096, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.24)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 35
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (8, 5), deadline = 20
Simulating trial. . .
epsilon = 0.7047; alpha = 0.5000
Simulating trial. . .
epsilon = 0.7047; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: forward, reward: -9.67791539744
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -9.677915397435696, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.68)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: forward, reward: -9.60198298351
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -9.601982983513581, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.60)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 2.85001066109
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.8500106610881915, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.85)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 2.23443464407
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.234434644067794, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.23)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 1.06233589666
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.0623358966649852, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.06)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: right, reward: 1.84496102567
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.844961025668884, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.84)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 2.79942956548
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.799429565484979, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.80)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: right, reward: 0.687133937891
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.6871339378912749, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.69)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: None, reward: 1.52575880104
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.525758801038395, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.53)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: left, reward: 2.39966590859
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 2.399665908588661, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.40)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: -40.6827721616
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': -40.68277216157401, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.68)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 0.21319508349
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.21319508348962213, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.21)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 0.85710556571
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.8571055657100484, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 0.86)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 1.88732566329
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.887325663294899, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.89)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: 1.02433599348
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', 'left'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.0243359934823733, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', 'left')
Agent followed the waypoint right. (rewarded 1.02)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 1.81084089303
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 1.8108408930315907, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent followed the waypoint forward. (rewarded 1.81)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 36
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (2, 6), deadline = 30
Simulating trial. . .
epsilon = 0.6977; alpha = 0.5000
Simulating trial. . .
epsilon = 0.6977; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: -10.9187456868
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': -10.918745686751345, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -10.92)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 2.32827319691
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.328273196906135, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.33)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 1.63951271557
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.6395127155716807, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.64)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: left, reward: 2.75948801794
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 27, 't': 3, 'action': 'left', 'reward': 2.759488017942483, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 2.76)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 1.10546565893
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 1.1054656589313039, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 1.11)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 0.0983397542901
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 0.0983397542901222, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent drove right instead of forward. (rewarded 0.10)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: 1.62792849097
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 1.6279284909683276, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 1.63)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 2.10506951617
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 2.1050695161692223, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', None)
Agent followed the waypoint forward. (rewarded 2.11)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: left, reward: -9.57312037328
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': -9.573120373280608, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.57)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 1.66320183335
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 1.6632018333456773, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 1.66)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: -4.43780688241
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 20, 't': 10, 'action': None, 'reward': -4.437806882410657, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.44)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: -4.56453096677
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'right'), 'deadline': 19, 't': 11, 'action': None, 'reward': -4.564530966772555, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.56)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: right, reward: 1.07958263391
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'left'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 1.0795826339084096, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'left')
Agent drove right instead of left. (rewarded 1.08)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 0.223484000682
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 0.22348400068248897, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.22)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 2.25751238982
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'forward'), 'deadline': 16, 't': 14, 'action': None, 'reward': 2.2575123898242757, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.26)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 1.36434740118
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.3643474011779135, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.36)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: left, reward: -19.840730017
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 14, 't': 16, 'action': 'left', 'reward': -19.840730016976156, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.84)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 2.19732767947
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': 2.19732767947429, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.20)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 0.398905070184
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 0.3989050701836572, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.40)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: left, reward: -10.0602230296
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 11, 't': 19, 'action': 'left', 'reward': -10.060223029562462, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.06)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: -40.3757118552
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 10, 't': 20, 'action': 'forward', 'reward': -40.3757118552425, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.38)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: -10.2350704035
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': -10.235070403520943, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.24)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 0.733428217536
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 8, 't': 22, 'action': 'left', 'reward': 0.7334282175355515, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 0.73)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: right, reward: -0.294005528079
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 7, 't': 23, 'action': 'right', 'reward': -0.2940055280787327, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent drove right instead of forward. (rewarded -0.29)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 0.769269802129
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 0.7692698021291786, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.77)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: -10.7853837839
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 5, 't': 25, 'action': 'forward', 'reward': -10.785383783859412, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -10.79)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 0.603437609651
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 4, 't': 26, 'action': None, 'reward': 0.6034376096509986, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 0.60)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 0.662063574716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 3, 't': 27, 'action': None, 'reward': 0.6620635747163, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.66)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: -5.26423544688
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 2, 't': 28, 'action': None, 'reward': -5.264235446879704, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.26)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: left, reward: 0.795464447383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 1, 't': 29, 'action': 'left', 'reward': 0.7954644473832226, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 0.80)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 37
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (2, 6), deadline = 30
Simulating trial. . .
epsilon = 0.6907; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 1.23624909571
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'left'), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.2362490957089338, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.24)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 0.718009623
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'left'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 0.7180096230000247, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'left')
Agent drove right instead of left. (rewarded 0.72)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: left, reward: -9.36619757929
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': 'left', 'reward': -9.366197579291216, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.37)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: 2.15319220783
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 2.153192207828173, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.15)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 0.0473192323818
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 0.04731923238179647, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.05)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: right, reward: 1.65766801151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.6576680115084237, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 1.66)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 1.88800233611
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.8880023361077067, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.89)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: right, reward: 1.6254087597
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.62540875969627, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent drove right instead of left. (rewarded 1.63)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: 2.39955991999
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.399559919993928, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.40)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: forward, reward: -0.0883516866259
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': -0.08835168662588111, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded -0.09)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: 1.1092574455
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 1.1092574454980153, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 1.11)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: right, reward: 0.9331827656
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 0.9331827655996449, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.93)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 2.57176317863
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.571763178626128, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.57)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: forward, reward: 0.722481729316
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 0.7224817293157475, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent drove forward instead of left. (rewarded 0.72)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: forward, reward: -10.8230679211
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': -10.823067921062572, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.82)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: forward, reward: -0.0311155633895
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': -0.03111556338952781, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove forward instead of left. (rewarded -0.03)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: forward, reward: 0.0231571622117
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': 0.02315716221166908, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.02)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 0.507728457161
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 0.5077284571610716, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.51)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.12545558336
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 12, 't': 18, 'action': None, 'reward': 2.1254555833612896, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.13)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: right, reward: 1.19210775149
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 1.1921077514899014, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.19)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 1.73860158559
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 10, 't': 20, 'action': None, 'reward': 1.73860158558594, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.74)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 0.707457555044
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 0.7074575550438803, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.71)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 0.143779911114
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 0.14377991111422372, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.14)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: -4.80872986513
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 23, 'action': None, 'reward': -4.808729865125664, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.81)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 0.573357694093
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 0.5733576940932268, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.57)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 0.924514437587
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 5, 't': 25, 'action': None, 'reward': 0.9245144375866108, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.92)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.19136908119
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 4, 't': 26, 'action': None, 'reward': 2.191369081191304, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.19)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 1.89525109568
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 27, 'action': 'forward', 'reward': 1.895251095679285, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.90)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 0.806567031621
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 2, 't': 28, 'action': None, 'reward': 0.8065670316211415, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.81)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 0.176108664452
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 1, 't': 29, 'action': 'right', 'reward': 0.17610866445208262, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent drove right instead of forward. (rewarded 0.18)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 38
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (1, 2), deadline = 25
Simulating trial. . .
epsilon = 0.6839; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 2.89682510562
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.8968251056199437, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.90)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: -20.5211841108
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': -20.521184110810918, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.52)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: -9.49179749455
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': -9.491797494554431, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.49)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.13955791135
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.1395579113510967, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.14)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: -40.0618856866
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 21, 't': 4, 'action': 'left', 'reward': -40.061885686648765, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.06)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 2.64767127792
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 2.647671277915326, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.65)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: left, reward: -10.9164910097
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': -10.91649100971619, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -10.92)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: left, reward: 0.534890095049
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 0.5348900950493849, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.53)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: left, reward: -9.77271504358
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 17, 't': 8, 'action': 'left', 'reward': -9.772715043582087, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent attempted driving left through a red light. (rewarded -9.77)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 0.677594491043
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 0.6775944910432528, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.68)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: -5.05591862008
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 15, 't': 10, 'action': None, 'reward': -5.055918620080218, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.06)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: -10.4835039997
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': -10.483503999650619, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent attempted driving forward through a red light. (rewarded -10.48)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 1.22446874364
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.2244687436403277, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.22)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: -4.42000442665
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 12, 't': 13, 'action': None, 'reward': -4.420004426652342, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.42)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: 0.716543890799
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 0.7165438907986281, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 0.72)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: -9.68833219202
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': -9.68833219201781, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.69)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 0.475533011399
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', 'forward'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 0.4755330113989983, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', 'forward')
Agent drove right instead of forward. (rewarded 0.48)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: -39.9579595107
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': -39.95795951070404, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.96)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: -9.19483391489
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'right'), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': -9.194833914893445, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'right')
Agent attempted driving forward through a red light. (rewarded -9.19)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: -0.475370816056
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'right'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': -0.47537081605569476, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'right')
Agent drove right instead of left. (rewarded -0.48)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: 1.4245341265
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 1.4245341265031117, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.42)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: 2.08201498676
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 2.0820149867612816, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.08)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: -9.94668611104
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': -9.946686111037673, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.95)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: -9.92427538873
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 2, 't': 23, 'action': 'forward', 'reward': -9.92427538872892, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.92)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 0.653042560708
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.6530425607080421, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 0.65)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 39
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (2, 5), deadline = 35
Simulating trial. . .
epsilon = 0.6771; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: forward, reward: 1.59830796782
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 35, 't': 0, 'action': 'forward', 'reward': 1.598307967817469, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.60)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: 1.38363865059
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 34, 't': 1, 'action': 'left', 'reward': 1.3836386505885385, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent followed the waypoint left. (rewarded 1.38)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: -10.1709207826
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 33, 't': 2, 'action': 'forward', 'reward': -10.170920782643174, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -10.17)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: -9.51089375264
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 32, 't': 3, 'action': 'forward', 'reward': -9.510893752637962, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.51)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: 2.2120923977
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 31, 't': 4, 'action': 'forward', 'reward': 2.2120923977031293, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.21)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: -5.2340455593
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 30, 't': 5, 'action': None, 'reward': -5.234045559297465, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.23)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 2.32214934919
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 29, 't': 6, 'action': 'forward', 'reward': 2.3221493491882885, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.32)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.46822138462
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 28, 't': 7, 'action': None, 'reward': 2.468221384624267, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.47)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.83757290709
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 27, 't': 8, 'action': None, 'reward': 2.837572907091437, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.84)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.95915808346
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 26, 't': 9, 'action': None, 'reward': 1.959158083461066, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.96)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: 0.247957493759
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 25, 't': 10, 'action': 'left', 'reward': 0.24795749375931886, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 0.25)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 1.73703084251
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 11, 'action': 'right', 'reward': 1.7370308425139576, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.74)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: -5.44396765161
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 23, 't': 12, 'action': None, 'reward': -5.44396765161123, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.44)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: -40.5336692934
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 22, 't': 13, 'action': 'forward', 'reward': -40.533669293382886, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.53)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.70780734257
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 21, 't': 14, 'action': None, 'reward': 1.7078073425731557, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.71)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 2.54787447096
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 15, 'action': None, 'reward': 2.5478744709606076, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.55)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: -9.05842152753
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 19, 't': 16, 'action': 'left', 'reward': -9.05842152753469, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent attempted driving left through a red light. (rewarded -9.06)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 1.67811066268
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 18, 't': 17, 'action': 'left', 'reward': 1.6781106626802256, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.68)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 2.09741192146
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 17, 't': 18, 'action': None, 'reward': 2.097411921456877, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.10)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 2.25054748502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 16, 't': 19, 'action': None, 'reward': 2.2505474850194687, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.25)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: -10.1768573253
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 20, 'action': 'left', 'reward': -10.176857325271232, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.18)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.22650680625
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 21, 'action': None, 'reward': 1.2265068062520204, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.23)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 0.182826116087
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 13, 't': 22, 'action': None, 'reward': 0.18282611608744292, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.18)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 0.485681940034
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', 'forward'), 'deadline': 12, 't': 23, 'action': 'right', 'reward': 0.48568194003361764, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', 'forward')
Agent drove right instead of forward. (rewarded 0.49)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 0.947415979187
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 11, 't': 24, 'action': None, 'reward': 0.9474159791867358, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 0.95)
29% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: 0.805269387173
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 10, 't': 25, 'action': 'right', 'reward': 0.8052693871725363, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 0.81)
26% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: -0.250896298309
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 9, 't': 26, 'action': 'forward', 'reward': -0.25089629830930016, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent drove forward instead of right. (rewarded -0.25)
23% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: left, reward: 1.08600198148
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 8, 't': 27, 'action': 'left', 'reward': 1.0860019814765063, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove left instead of right. (rewarded 1.09)
20% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 0.434034395806
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 7, 't': 28, 'action': 'right', 'reward': 0.434034395805865, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.43)
17% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 1.72871584966
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 6, 't': 29, 'action': 'right', 'reward': 1.7287158496648045, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.73)
14% of time remaining to reach destination.
/-------------------
| Step 30 Results
\-------------------
Environment.step(): t = 30
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 0.634501809259
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 5, 't': 30, 'action': None, 'reward': 0.6345018092587051, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 0.63)
11% of time remaining to reach destination.
/-------------------
| Step 31 Results
\-------------------
Environment.step(): t = 31
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: -9.22836111254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 4, 't': 31, 'action': 'forward', 'reward': -9.228361112536032, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.23)
9% of time remaining to reach destination.
/-------------------
| Step 32 Results
\-------------------
Environment.step(): t = 32
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: -10.9522624321
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 3, 't': 32, 'action': 'left', 'reward': -10.952262432080651, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.95)
6% of time remaining to reach destination.
/-------------------
| Step 33 Results
\-------------------
Environment.step(): t = 33
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: right, reward: 0.814027276191
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 2, 't': 33, 'action': 'right', 'reward': 0.8140272761909879, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 0.81)
3% of time remaining to reach destination.
/-------------------
| Step 34 Results
\-------------------
Environment.step(): t = 34
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: left, reward: -40.1536452246
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 1, 't': 34, 'action': 'left', 'reward': -40.153645224575314, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.15)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 40
\-------------------------
Environment.reset(): Trial set up with start = (4, 7), destination = (8, 6), deadline = 25
Simulating trial. . .
epsilon = 0.6703; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.89010763795
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.8901076379542416, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.89)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 2.10326907117
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.103269071165766, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.10)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.56481290194
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.5648129019383115, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.56)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.13530214675
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.135302146752828, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.14)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 1.6280892594
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 1.6280892594006284, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.63)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.92039164224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.920391642235809, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.92)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: -20.7078808484
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': -20.707880848352858, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.71)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: -10.441928732
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': -10.441928732024273, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.44)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: -10.6344569569
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': -10.634456956853121, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.63)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: left, reward: 0.65328612272
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 0.6532861227198681, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.65)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: forward, reward: -9.41698857444
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': -9.41698857443518, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent attempted driving forward through a red light. (rewarded -9.42)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 1.27256484122
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.272564841220154, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.27)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 1.70136384496
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.7013638449598996, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.70)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: 0.831379944322
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 0.8313799443222625, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.83)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: left, reward: -39.8895644825
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -39.889564482496, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.89)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 1.57619096789
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.5761909678899646, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.58)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: -0.0110877039165
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 9, 't': 16, 'action': 'right', 'reward': -0.01108770391645797, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded -0.01)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: -4.00401454412
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 8, 't': 17, 'action': None, 'reward': -4.0040145441160835, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.00)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: left, reward: 1.40637063392
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 18, 'action': 'left', 'reward': 1.4063706339165847, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.41)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 1.31793056377
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 19, 'action': None, 'reward': 1.317930563770757, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.32)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 2.25125559804
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 2.2512555980423947, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.25)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: forward, reward: -0.366502700748
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', None), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': -0.3665027007476648, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', None)
Agent drove forward instead of left. (rewarded -0.37)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 0.671381442309
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.6713814423092046, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.67)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 0.619872467434
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 2, 't': 23, 'action': None, 'reward': 0.6198724674341258, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.62)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: left, reward: -9.6062152736
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 1, 't': 24, 'action': 'left', 'reward': -9.60621527359665, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent attempted driving left through a red light. (rewarded -9.61)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 41
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (5, 3), deadline = 30
Simulating trial. . .
epsilon = 0.6637; alpha = 0.5000
Simulating trial. . .
epsilon = 0.6637; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 1.75318793602
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.7531879360219471, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.75)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: -4.61048552553
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 29, 't': 1, 'action': None, 'reward': -4.610485525531639, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.61)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 0.13903418484
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 0.13903418483951124, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove right instead of left. (rewarded 0.14)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: 2.22652142768
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 27, 't': 3, 'action': 'left', 'reward': 2.226521427681212, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.23)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: -20.7037166484
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 26, 't': 4, 'action': 'left', 'reward': -20.703716648403248, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.70)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: -5.34674571582
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 25, 't': 5, 'action': None, 'reward': -5.3467457158162, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.35)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 0.988673730776
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 0.9886737307756985, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.99)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 0.930644839518
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 0.9306448395178574, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 0.93)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: left, reward: -39.3927743814
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 22, 't': 8, 'action': 'left', 'reward': -39.39277438143367, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.39)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 1.04137096934
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 1.0413709693366353, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.04)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: left, reward: 0.874694138353
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 0.874694138353197, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.87)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 1.30096453883
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', 'left'), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 1.300964538834442, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', 'left')
Agent drove right instead of forward. (rewarded 1.30)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: -5.51963204624
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'forward'), 'deadline': 18, 't': 12, 'action': None, 'reward': -5.519632046235164, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.52)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: forward, reward: -39.1249335564
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': -39.12493355638631, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.12)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: forward, reward: -39.1055381918
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', None), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': -39.10553819175735, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.11)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: left, reward: -10.5167471114
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 15, 't': 15, 'action': 'left', 'reward': -10.516747111437034, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.52)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: left, reward: -10.049471737
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': -10.04947173695787, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.05)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: right, reward: 1.09411605068
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 1.0941160506768413, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 1.09)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 1.78011165817
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.7801116581747516, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.78)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: 0.566768459291
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': 0.5667684592914412, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.57)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: -20.5381450571
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 10, 't': 20, 'action': 'left', 'reward': -20.538145057061907, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.54)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 0.928992036446
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 21, 'action': None, 'reward': 0.9289920364464175, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.93)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: -10.0944951478
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 22, 'action': 'left', 'reward': -10.09449514775568, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.09)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 0.816297803
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 0.8162978030003133, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.82)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: left, reward: 1.59338144824
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 6, 't': 24, 'action': 'left', 'reward': 1.5933814482383202, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.59)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: left, reward: -40.0869503445
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 5, 't': 25, 'action': 'left', 'reward': -40.08695034454693, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.09)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 0.490214961118
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 4, 't': 26, 'action': None, 'reward': 0.49021496111810325, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 0.49)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 1.31121749319
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 3, 't': 27, 'action': 'forward', 'reward': 1.3112174931897078, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.31)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 0.424203049917
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 28, 'action': None, 'reward': 0.42420304991651103, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.42)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: -4.16333316683
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 1, 't': 29, 'action': None, 'reward': -4.163333166825884, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.16)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 42
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (6, 4), deadline = 20
Simulating trial. . .
epsilon = 0.6570; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: -20.2025482811
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'right'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -20.20254828113961, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.20)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: -5.17079083846
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 19, 't': 1, 'action': None, 'reward': -5.170790838460703, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.17)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: -5.73054158898
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': -5.7305415889844165, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.73)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.69248505327
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.6924850532680997, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.69)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: -10.8749486009
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -10.874948600872656, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent attempted driving left through a red light. (rewarded -10.87)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 1.64533801491
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.6453380149089485, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.65)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: 2.30098437973
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.3009843797267058, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.30)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: -5.6760419982
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', None), 'deadline': 13, 't': 7, 'action': None, 'reward': -5.676041998198667, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.68)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: forward, reward: 1.40821702519
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.4082170251873776, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 1.41)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: -5.27207037895
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': -5.272070378954165, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.27)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.52576227798
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.5257622779840583, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.53)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.62434974595
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.62434974595473, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.62)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.77695340238
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.7769534023808968, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.78)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: forward, reward: 2.04332626379
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 2.043326263786522, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.04)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 43
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (4, 3), deadline = 25
Simulating trial. . .
epsilon = 0.6505; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: 1.03793966619
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.0379396661889382, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.04)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: -9.98854772705
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', 'left'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -9.988547727050895, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', 'left')
Agent attempted driving left through a red light. (rewarded -9.99)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.964845663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.9648456630039766, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.96)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: -10.6518737827
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'left', 'reward': -10.651873782688572, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.65)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: 2.13057099304
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.1305709930444907, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent followed the waypoint forward. (rewarded 2.13)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 2.54938864223
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.5493886422305434, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.55)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: -10.8305921321
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -10.830592132080527, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.83)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: -9.48874989292
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 7, 'action': 'left', 'reward': -9.488749892920962, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.49)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.52272955812
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.522729558117577, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.52)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 0.361197632219
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.3611976322190553, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 0.36)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 2.00092965887
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.000929658869228, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.00)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: -5.59317490456
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': -5.593174904563494, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.59)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: forward, reward: 1.56221111776
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'right', 'right'), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 1.5622111177629416, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'right', 'right')
Agent drove forward instead of right. (rewarded 1.56)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: right, reward: 0.712450548422
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 0.7124505484219796, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove right instead of forward. (rewarded 0.71)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: left, reward: -19.3208125121
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -19.32081251205997, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.32)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: left, reward: 0.98852435366
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 0.9885243536596251, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 0.99)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: left, reward: 1.17455277503
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 1.1745527750265368, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.17)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: 1.63188275517
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 8, 't': 17, 'action': None, 'reward': 1.6318827551675177, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.63)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 2.37212678349
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 2.3721267834896267, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 2.37)
24% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 44
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (5, 4), deadline = 25
Simulating trial. . .
epsilon = 0.6440; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: right, reward: 2.44217403635
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.4421740363483346, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', 'forward')
Agent followed the waypoint right. (rewarded 2.44)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 1.19962079671
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.1996207967141723, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove right instead of forward. (rewarded 1.20)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: None, reward: 1.88897500583
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.8889750058309085, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.89)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: None, reward: 1.90089044885
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.9008904488476428, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.90)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: right, reward: 1.86638823793
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'right'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.8663882379331969, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'right')
Agent drove right instead of left. (rewarded 1.87)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: right, reward: -19.6565674321
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': -19.656567432085172, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.66)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: right, reward: 1.10313755414
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.1031375541413464, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.10)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: None, reward: -4.17227939131
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 18, 't': 7, 'action': None, 'reward': -4.172279391306191, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.17)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: None, reward: -4.56946969826
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': -4.569469698263819, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.57)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: left, reward: 1.14055036055
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.140550360547609, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent drove left instead of right. (rewarded 1.14)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: left, reward: -0.0487936485482
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': -0.04879364854818691, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove left instead of forward. (rewarded -0.05)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 1.27453161572
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.2745316157231272, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.27)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 2.69940902924
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 2.6994090292393924, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.70)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 1.95647508023
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.9564750802285626, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.96)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 2.28801593657
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.2880159365671564, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 2.29)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 1.92559523487
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.9255952348692493, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.93)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 1.488908613
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 1.4889086129987814, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.49)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: forward, reward: -10.7388754902
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': -10.738875490165428, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.74)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 0.869660264704
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 7, 't': 18, 'action': None, 'reward': 0.8696602647043645, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 0.87)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 0.106170441071
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 0.1061704410706793, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 0.11)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: -9.207678998
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 5, 't': 20, 'action': 'left', 'reward': -9.207678997997283, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent attempted driving left through a red light. (rewarded -9.21)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.28359081727
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'left'), 'deadline': 4, 't': 21, 'action': None, 'reward': 1.2835908172660844, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.28)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 0.259790565522
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': 0.25979056552227453, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.26)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 2.0166647595
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 2, 't': 23, 'action': 'forward', 'reward': 2.0166647595041134, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.02)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: -9.61621996125
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': -9.616219961251419, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -9.62)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 45
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (5, 6), deadline = 30
Simulating trial. . .
epsilon = 0.6376; alpha = 0.5000
Simulating trial. . .
epsilon = 0.6376; alpha = 0.5000
Simulating trial. . .
epsilon = 0.6376; alpha = 0.5000
Simulating trial. . .
epsilon = 0.6376; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.74682261437
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.7468226143721948, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.75)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 0.742547247405
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 0.7425472474047251, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.74)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: left, reward: 2.22889276732
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 28, 't': 2, 'action': 'left', 'reward': 2.228892767323841, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.23)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 2.09970962457
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.099709624574638, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.10)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 1.28261527932
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.2826152793181425, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.28)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 0.990320272783
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'right'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 0.9903202727827942, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'right')
Agent followed the waypoint forward. (rewarded 0.99)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: left, reward: 0.992973799755
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 0.9929737997553171, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.99)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: -4.2135690279
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 23, 't': 7, 'action': None, 'reward': -4.213569027898084, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.21)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 2.32734962382
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 2.3273496238191544, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.33)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: left, reward: -39.2345111498
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 21, 't': 9, 'action': 'left', 'reward': -39.234511149769055, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.23)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: 0.800167552266
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', 'left'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 0.8001675522661038, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', 'left')
Agent drove right instead of forward. (rewarded 0.80)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: left, reward: -10.6874541996
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': -10.68745419963818, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.69)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: left, reward: -9.5121272688
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': 'left', 'reward': -9.512127268797702, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.51)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: 2.61859063609
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 2.6185906360944444, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.62)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 2.65681776329
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 2.6568177632855483, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.66)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: -9.26511580829
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 15, 't': 15, 'action': 'left', 'reward': -9.265115808288398, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -9.27)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: right, reward: 1.30489892467
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 1.3048989246703693, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.30)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: forward, reward: 2.49148694128
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': 2.491486941278685, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.49)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: forward, reward: 1.53401528613
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'right'), 'deadline': 12, 't': 18, 'action': 'forward', 'reward': 1.534015286133125, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'right')
Agent followed the waypoint forward. (rewarded 1.53)
37% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 46
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (1, 7), deadline = 20
Simulating trial. . .
epsilon = 0.6313; alpha = 0.5000
Simulating trial. . .
epsilon = 0.6313; alpha = 0.5000
Simulating trial. . .
epsilon = 0.6313; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: -5.36667640194
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': -5.36667640194249, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.37)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: 2.58848950189
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 2.588489501894293, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 2.59)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 2.23087969165
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.230879691646685, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.23)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 2.38816142086
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.3881614208601336, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.39)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: -10.9375357932
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -10.937535793174144, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.94)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.82946832572
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.8294683257242577, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.83)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: 0.64856678732
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.6485667873204686, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.65)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: -4.41880960426
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': -4.418809604263438, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.42)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 0.352578745951
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 0.35257874595063654, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.35)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 1.73662454383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.7366245438304015, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.74)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 0.188425692415
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.18842569241479412, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent drove right instead of left. (rewarded 0.19)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: forward, reward: -0.046603305734
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'forward'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': -0.04660330573400895, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'forward')
Agent drove forward instead of right. (rewarded -0.05)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: 1.52011313994
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'right', 'right'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 1.5201131399398786, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'right', 'right')
Agent drove forward instead of right. (rewarded 1.52)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: left, reward: -39.1324821952
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': -39.13248219520342, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.13)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 2.46454042896
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.4645404289550124, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.46)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.33473903517
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.3347390351652113, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.33)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: left, reward: 0.659190875871
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 0.6591908758707892, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.66)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: left, reward: 0.4491309604
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 0.4491309604003141, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 0.45)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: left, reward: -40.3902407816
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -40.390240781570306, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.39)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: forward, reward: 0.983702561948
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 0.9837025619482596, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.98)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 47
\-------------------------
Environment.reset(): Trial set up with start = (3, 4), destination = (1, 6), deadline = 20
Simulating trial. . .
epsilon = 0.6250; alpha = 0.5000
Simulating trial. . .
epsilon = 0.6250; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 1.06586863042
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.0658686304244704, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.07)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: -4.00305937986
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': -4.003059379858582, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.00)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 2.7794767647
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 2.7794767646956173, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.78)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.27764904682
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.2776490468154758, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.28)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.23484932341
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.2348493234138376, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.23)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 1.34071731841
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'right', 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.3407173184115957, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'right', 'forward')
Agent drove right instead of left. (rewarded 1.34)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: forward, reward: -9.23968356918
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -9.239683569184399, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.24)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: 1.12073218237
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.1207321823707368, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.12)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: 1.30017468501
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.3001746850148725, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.30)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 0.83327914182
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.8332791418201142, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 0.83)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.19856343815
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.1985634381476804, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.20)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: -4.0564281567
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 9, 't': 11, 'action': None, 'reward': -4.056428156702776, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.06)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 0.925407867451
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.9254078674513182, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.93)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 0.961928987476
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 0.9619289874758019, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 0.96)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 2.39829986939
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.398299869390173, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.40)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: left, reward: 2.18468759485
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 2.184687594849817, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.18)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 48
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (4, 3), deadline = 20
Simulating trial. . .
epsilon = 0.6188; alpha = 0.5000
Simulating trial. . .
epsilon = 0.6188; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: 1.37874393225
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.3787439322509105, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.38)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: 1.94077701339
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'left', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.940777013391419, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'left', None)
Agent followed the waypoint right. (rewarded 1.94)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: -10.6374449739
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': -10.637444973926627, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.64)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: -10.6534087747
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': -10.653408774677427, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.65)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 2.26252234218
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.262522342182857, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.26)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: -10.3435882107
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': -10.34358821069831, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.34)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: forward, reward: 1.70192114374
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.7019211437354884, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded 1.70)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: 2.5374344653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.5374344653002177, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 2.54)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: right, reward: 0.443605415212
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.4436054152123291, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 0.44)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: 1.46691804994
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.4669180499441614, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.47)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: forward, reward: -10.2324124978
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': -10.232412497776666, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.23)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: forward, reward: -10.2042846141
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': -10.204284614054867, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.20)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: 0.765164606242
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.7651646062416468, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.77)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: 1.85907051489
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.8590705148865032, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.86)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: -9.19061999676
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -9.190619996762251, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.19)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: -9.41134564639
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -9.41134564639294, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -9.41)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 0.856243544545
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.8562435445452696, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.86)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: left, reward: 1.0012546721
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 1.0012546721011102, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.00)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: right, reward: 1.49507179108
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 1.495071791083808, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.50)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: forward, reward: -0.556963557878
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -0.5569635578781776, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded -0.56)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 49
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (7, 5), deadline = 30
Simulating trial. . .
epsilon = 0.6126; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.32479728815
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'forward'), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.3247972881527421, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.32)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 0.312875143425
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 0.3128751434248357, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 0.31)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.42417771174
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.4241777117378156, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.42)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: forward, reward: -10.007810955
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': -10.007810954962723, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.01)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: left, reward: -10.7919824283
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 26, 't': 4, 'action': 'left', 'reward': -10.791982428283658, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent attempted driving left through a red light. (rewarded -10.79)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: -5.11969066029
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': -5.11969066028736, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.12)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 0.236251923922
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 0.23625192392232164, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.24)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 1.62441984386
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.6244198438553288, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.62)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: -9.0260408618
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': -9.026040861799494, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.03)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: left, reward: 2.76863854988
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 2.7686385498833603, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.77)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 1.67628600964
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.6762860096403636, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.68)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: 2.04268200528
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 2.0426820052825043, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.04)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.29532082763
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.2953208276262775, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.30)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 1.51427518748
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 1.51427518747883, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.51)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: 2.09486693981
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 14, 'action': 'left', 'reward': 2.0948669398117765, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.09)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: 1.44576063723
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': 1.4457606372261491, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 1.45)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: -10.9240682263
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 14, 't': 16, 'action': 'left', 'reward': -10.924068226285755, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.92)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: -0.111356404025
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'right', 'left'), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': -0.11135640402482583, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'right', 'left')
Agent drove forward instead of left. (rewarded -0.11)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 0.77161163259
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 18, 'action': None, 'reward': 0.7716116325902129, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.77)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 0.870138596653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 19, 'action': None, 'reward': 0.87013859665327, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.87)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 0.798107143607
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 10, 't': 20, 'action': None, 'reward': 0.7981071436069531, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.80)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 1.55286043494
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'forward'), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 1.5528604349404205, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'forward')
Agent drove right instead of left. (rewarded 1.55)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: left, reward: 0.564666938608
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 8, 't': 22, 'action': 'left', 'reward': 0.5646669386081387, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove left instead of right. (rewarded 0.56)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: left, reward: -9.41431604795
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 7, 't': 23, 'action': 'left', 'reward': -9.414316047945945, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent attempted driving left through a red light. (rewarded -9.41)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 2.27966335222
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 2.2796633522217853, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 2.28)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 1.37379461705
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 1.373794617053207, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.37)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: left, reward: -0.0602800130536
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 4, 't': 26, 'action': 'left', 'reward': -0.06028001305363917, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded -0.06)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: None, reward: 0.675083446197
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 3, 't': 27, 'action': None, 'reward': 0.6750834461970814, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.68)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: left, reward: 0.903779588947
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 2, 't': 28, 'action': 'left', 'reward': 0.9037795889469981, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove left instead of right. (rewarded 0.90)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 1.67891023626
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 1, 't': 29, 'action': 'right', 'reward': 1.6789102362577155, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.68)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 50
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (7, 3), deadline = 20
Simulating trial. . .
epsilon = 0.6065; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: left, reward: 1.0553889894
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.055388989400735, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 1.06)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: forward, reward: -39.0688886854
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'right'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -39.06888868538704, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'right')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.07)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 1.73277397737
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'right'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.732773977373902, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'right')
Agent properly idled at a red light. (rewarded 1.73)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 2.06345393919
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'left', 'forward'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 2.0634539391889053, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.06)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: left, reward: -10.9081250656
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -10.908125065613358, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.91)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 0.702368388045
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 0.7023683880449794, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.70)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 0.74599738068
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 0.745997380679994, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.75)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 0.565243214665
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.5652432146648023, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 0.57)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 0.0671547573608
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.0671547573607717, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.07)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: -4.65806253455
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': -4.658062534550438, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.66)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 0.627515736032
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.6275157360320526, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent drove right instead of left. (rewarded 0.63)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 2.59798426112
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 2.597984261120904, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 2.60)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 2.33996143211
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.3399614321098343, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.34)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 0.0278312987608
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.027831298760760403, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.03)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: left, reward: 1.39932586143
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 1.3993258614285342, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 1.40)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: left, reward: -10.4182167207
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -10.418216720670097, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent attempted driving left through a red light. (rewarded -10.42)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: left, reward: 1.06202114971
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 1.0620211497079661, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 1.06)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 1.59992785934
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 1.5999278593369637, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.60)
10% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 51
\-------------------------
Environment.reset(): Trial set up with start = (7, 2), destination = (5, 5), deadline = 25
Simulating trial. . .
epsilon = 0.6005; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: forward, reward: -9.58701336544
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'right', 'left'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -9.587013365441464, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', 'left')
Agent attempted driving forward through a red light. (rewarded -9.59)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: left, reward: -9.19117864347
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -9.19117864347237, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent attempted driving left through a red light. (rewarded -9.19)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 1.26765606565
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.267656065653188, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.27)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: forward, reward: -9.61042597482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': -9.610425974822578, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.61)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: forward, reward: -39.579128987
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -39.57912898699861, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.58)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 2.73300561993
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.7330056199292683, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.73)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: 1.23044161507
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.2304416150683724, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 1.23)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 0.0828376286271
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.0828376286271344, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.08)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 0.701436787565
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.70143678756497, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.70)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 1.37651269709
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.3765126970947374, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent drove right instead of left. (rewarded 1.38)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: -4.05008727582
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 15, 't': 10, 'action': None, 'reward': -4.050087275823692, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.05)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.09182184225
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.091821842247174, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.09)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: right, reward: 2.12318735327
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 2.12318735326824, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.12)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: left, reward: 0.618888740012
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 0.6188887400124998, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.62)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: left, reward: -39.6937407156
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -39.693740715615206, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.69)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: left, reward: -39.7902227976
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 10, 't': 15, 'action': 'left', 'reward': -39.79022279759712, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.79)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: left, reward: -19.9784315131
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, 'left', 'right'), 'deadline': 9, 't': 16, 'action': 'left', 'reward': -19.97843151314959, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.98)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: 0.375150768185
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 0.37515076818500415, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove left instead of right. (rewarded 0.38)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: -40.7765638536
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 7, 't': 18, 'action': 'left', 'reward': -40.77656385361018, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.78)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: left, reward: 0.27243283189
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 0.27243283188970036, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent drove left instead of right. (rewarded 0.27)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: -5.67218410453
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 5, 't': 20, 'action': None, 'reward': -5.672184104531511, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.67)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: left, reward: -9.09297698745
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': -9.092976987451522, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -9.09)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: left, reward: -40.1589267285
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 3, 't': 22, 'action': 'left', 'reward': -40.15892672848731, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.16)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 1.93606506715
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.9360650671533948, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.94)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 1.98685049267
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.986850492665822, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.99)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 52
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (6, 3), deadline = 25
Simulating trial. . .
epsilon = 0.5945; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5945; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5945; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5945; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: -19.236169017
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'left'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': -19.236169016983048, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.24)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: right, reward: 0.6520679159
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.6520679158996039, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 0.65)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 2.04730934653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.047309346529909, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.05)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 1.74547593276
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.745475932763353, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.75)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: right, reward: 1.98963827988
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.989638279880404, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.99)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: left, reward: -10.5557769125
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'left', 'reward': -10.555776912491558, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.56)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: -5.89000423135
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 19, 't': 6, 'action': None, 'reward': -5.89000423134803, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.89)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: forward, reward: 1.60447243796
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.6044724379576951, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.60)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: right, reward: 0.40303738746
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.40303738745982776, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent drove right instead of left. (rewarded 0.40)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: right, reward: 1.2087616064
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.2087616064000914, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.21)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: right, reward: 0.0452876413283
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.0452876413283253, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent drove right instead of left. (rewarded 0.05)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: left, reward: 1.31398841415
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.3139884141540144, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent drove left instead of right. (rewarded 1.31)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: 0.51540997663
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 0.5154099766295566, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.52)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: -9.63722384452
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': -9.637223844517635, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -9.64)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 2.16308071873
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 2.163080718730585, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.16)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: left, reward: -0.184394304156
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': -0.1843943041558509, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove left instead of right. (rewarded -0.18)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: -9.77860070401
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': -9.778600704010993, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.78)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 0.337796181593
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 0.3377961815930628, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.34)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: left, reward: 0.636281168556
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 18, 'action': 'left', 'reward': 0.6362811685559979, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.64)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 2.23131352208
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', 'forward'), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 2.2313135220827567, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', 'forward')
Agent followed the waypoint forward. (rewarded 2.23)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: left, reward: -10.7360790213
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 5, 't': 20, 'action': 'left', 'reward': -10.736079021278432, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent attempted driving left through a red light. (rewarded -10.74)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: -4.83916582692
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': None, 'reward': -4.839165826920198, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.84)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: -5.37915199976
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 22, 'action': None, 'reward': -5.379151999762897, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.38)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: -10.2483306815
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 2, 't': 23, 'action': 'forward', 'reward': -10.24833068147957, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.25)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 1.48268672425
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.4826867242510506, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.48)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 53
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (4, 4), deadline = 20
Simulating trial. . .
epsilon = 0.5886; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5886; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 1.60441829105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.60441829104704, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.60)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: left, reward: -9.624732393
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -9.624732392999459, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.62)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 1.10803442231
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.108034422314416, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.11)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 2.90518308402
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 2.9051830840177875, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.91)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.54336605888
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.5433660588832452, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.54)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.08803332625
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.0880333262530022, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.09)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.89049752962
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.8904975296165947, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.89)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: 1.3725422033
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.3725422032981918, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove right instead of forward. (rewarded 1.37)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 1.63188062225
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.6318806222510451, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.63)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: forward, reward: -9.90228497909
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': -9.902284979090204, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.90)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 1.62985493844
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.6298549384350143, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.63)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: left, reward: -19.9579271099
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': -19.957927109912678, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.96)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: left, reward: 0.941721810793
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 0.9417218107926626, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 0.94)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 0.717094061587
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 0.7170940615865293, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 0.72)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: left, reward: -40.2171626094
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -40.217162609382754, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.22)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 0.994664752467
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.9946647524673937, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.99)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 0.38246645791
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.38246645790993017, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.38)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: -0.414626680028
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': -0.4146266800275785, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent drove right instead of left. (rewarded -0.41)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 0.847132452407
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.8471324524072172, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 0.85)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 0.991187586061
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.9911875860614321, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.99)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 54
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (4, 2), deadline = 20
Simulating trial. . .
epsilon = 0.5827; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: -9.87971971934
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -9.8797197193351, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.88)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.82337684436
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.8233768443565856, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.82)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.77930474284
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.7793047428394955, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.78)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: right, reward: 1.79315871139
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.7931587113916683, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.79)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 1.52456148579
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.5245614857903995, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.52)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 0.268446573705
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 0.26844657370521596, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.27)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 1.64369836996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.6436983699614416, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', None)
Agent drove right instead of left. (rewarded 1.64)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 1.10227821072
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.1022782107178777, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.10)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 1.16017839916
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.1601783991557957, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.16)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 1.42699690739
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.4269969073863227, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.43)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 1.14737543652
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.1473754365156028, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.15)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: 2.24425700343
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 2.244257003429407, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.24)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: -10.2777937049
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -10.277793704928886, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -10.28)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: 2.16456454903
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 2.16456454903157, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.16)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: -0.103765718005
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': -0.10376571800518508, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded -0.10)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: forward, reward: 0.231994666313
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 0.23199466631255117, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.23)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: left, reward: -40.9490824936
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': -40.949082493558684, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.95)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: -4.27370867903
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': -4.273708679026351, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.27)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: -0.0800776671505
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': -0.08007766715046838, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded -0.08)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: left, reward: 1.03351193276
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 1.0335119327556868, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 1.03)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 55
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (1, 4), deadline = 25
Simulating trial. . .
epsilon = 0.5769; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: -39.9737816643
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -39.973781664265424, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.97)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.5097820462
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.5097820462039402, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.51)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.72194494835
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.7219449483516431, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.72)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 1.0408761946
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.0408761945998106, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.04)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 2.49947724787
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.4994772478678726, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.50)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 0.209179241886
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.20917924188608772, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.21)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 1.13856945139
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.1385694513902451, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.14)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: forward, reward: -39.2014891698
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': -39.2014891697951, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.20)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 2.83367964313
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.8336796431322187, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.83)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: forward, reward: 1.82777228391
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.827772283909465, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.83)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 2.64184917337
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.6418491733673877, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.64)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 0.63979326808
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 0.6397932680803208, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.64)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 0.776438175573
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 0.7764381755726038, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent drove right instead of left. (rewarded 0.78)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 1.35156870453
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.3515687045294518, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.35)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: 1.19525184265
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 1.1952518426501353, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 1.20)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: 1.13459288192
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 1.1345928819222602, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove left instead of right. (rewarded 1.13)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 0.00858643145065
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 0.008586431450646193, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove right instead of left. (rewarded 0.01)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 1.37738151621
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.3773815162083023, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.38)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 1.54708653954
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': 1.5470865395428377, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.55)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 0.990856686578
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'forward'), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 0.9908566865777958, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.99)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: left, reward: -9.77042015354
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': 'left', 'reward': -9.77042015353734, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.77)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: left, reward: 0.633995304538
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 0.6339953045378555, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.63)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 0.181803438145
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.1818034381445195, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 0.18)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.51919193071
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.519191930707642, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.52)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: -0.902164763444
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 1, 't': 24, 'action': 'right', 'reward': -0.9021647634440606, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded -0.90)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 56
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (2, 2), deadline = 20
Simulating trial. . .
epsilon = 0.5712; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 1.63184965143
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.6318496514274285, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.63)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: forward, reward: -10.6527950725
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -10.65279507253336, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -10.65)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 2.92454535883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.924545358827762, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.92)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 2.35989836005
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.359898360050898, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.36)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 0.460917824731
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 0.4609178247314051, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.46)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 1.12395639956
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.1239563995594035, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.12)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: -10.842049734
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -10.84204973397472, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.84)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 1.72302290985
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.7230229098521739, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.72)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: left, reward: -10.3594977806
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': -10.35949778058199, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.36)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 0.875277581495
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.8752775814952729, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.88)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 1.33413386935
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.3341338693486766, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', None)
Agent drove right instead of forward. (rewarded 1.33)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 1.90692225864
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.906922258644846, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.91)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 1.81741509337
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.817415093367279, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.82)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: left, reward: -20.5989833295
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': -20.59898332945075, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.60)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 0.827227852284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.8272278522840243, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.83)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: right, reward: 1.10619370017
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.106193700173825, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent drove right instead of left. (rewarded 1.11)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: 1.85551988233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.8555198823279653, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.86)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: right, reward: 1.23812603255
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 1.2381260325488683, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.24)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: forward, reward: 0.768425350743
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 0.7684253507429715, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 0.77)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 1.93278910405
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 1.932789104053928, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.93)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 57
\-------------------------
Environment.reset(): Trial set up with start = (2, 2), destination = (7, 3), deadline = 20
Simulating trial. . .
epsilon = 0.5655; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5655; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5655; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5655; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 2.09234294791
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.0923429479098132, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.09)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: left, reward: 0.790309075586
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 0.7903090755860979, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 0.79)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 1.55214124329
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.552141243291818, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.55)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: left, reward: -9.8184669863
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -9.818466986304998, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.82)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 2.5577409304
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.5577409304020944, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.56)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.80910865263
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.8091086526293292, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.81)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 0.725188432003
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.7251884320027611, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.73)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: -39.5588706861
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -39.55887068610419, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.56)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 0.854809924838
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 0.8548099248377283, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.85)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: 1.01180354983
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.0118035498348212, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 1.01)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: left, reward: 0.828864559802
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 0.8288645598019678, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove left instead of right. (rewarded 0.83)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: forward, reward: 1.0418493778
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'right'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.04184937780415, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'right')
Agent drove forward instead of left. (rewarded 1.04)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 0.560935954108
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.5609359541077792, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.56)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 1.38821720627
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.3882172062701703, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.39)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: -9.10468115178
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -9.10468115177685, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.10)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: -9.51026375342
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -9.510263753416565, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.51)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: -10.7602490562
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': 'left', 'reward': -10.760249056220571, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.76)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: -0.312714804331
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', 'right'), 'deadline': 3, 't': 17, 'action': 'right', 'reward': -0.31271480433063414, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', 'right')
Agent drove right instead of forward. (rewarded -0.31)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: left, reward: 2.18321750517
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': 2.183217505169375, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.18)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 1.06026991275
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.0602699127520903, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.06)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 58
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (5, 7), deadline = 20
Simulating trial. . .
epsilon = 0.5599; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5599; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 0.720296870216
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.7202968702163459, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.72)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: right, reward: 1.30926342104
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.3092634210351792, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove right instead of left. (rewarded 1.31)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: right, reward: 1.67931164106
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.6793116410611755, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 1.68)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 1.48542162076
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.4854216207618882, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.49)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 0.320069947188
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.32006994718835113, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove right instead of forward. (rewarded 0.32)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: left, reward: -10.2665832382
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': -10.2665832381724, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -10.27)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 1.59453918451
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.5945391845066965, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.59)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: None, reward: 2.53975081545
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.539750815447377, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 2.54)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: right, reward: 1.59077056602
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.590770566018933, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.59)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: right, reward: 1.66830208047
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.6683020804722979, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 1.67)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: left, reward: -10.4960339041
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -10.49603390405599, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.50)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: forward, reward: 1.03892895227
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.0389289522718823, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'left')
Agent drove forward instead of right. (rewarded 1.04)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 0.851737422906
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.8517374229058381, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.85)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 2.58832513268
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 2.5883251326819265, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent followed the waypoint right. (rewarded 2.59)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 1.50450197719
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.5045019771900123, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent drove right instead of forward. (rewarded 1.50)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: left, reward: 1.46894379965
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 1.4689437996547372, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.47)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 1.64748391856
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.6474839185558896, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.65)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: -10.8857535574
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -10.885753557425533, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.89)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 0.744360211644
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.7443602116442762, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 0.74)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 1.05562357595
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.0556235759474994, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.06)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 59
\-------------------------
Environment.reset(): Trial set up with start = (5, 7), destination = (1, 3), deadline = 30
Simulating trial. . .
epsilon = 0.5543; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: 1.58542609109
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': 1.5854260910918152, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.59)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 0.195530991333
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 0.19553099133310126, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.20)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 0.606761941954
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 0.6067619419535369, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 0.61)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 2.23461833674
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.234618336735089, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.23)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: left, reward: -10.1505529179
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': 'left', 'reward': -10.15055291791899, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.15)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 0.321020936269
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 0.3210209362687443, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.32)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: left, reward: -10.2712956314
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 24, 't': 6, 'action': 'left', 'reward': -10.271295631412503, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.27)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: left, reward: -20.2590810073
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 23, 't': 7, 'action': 'left', 'reward': -20.259081007335105, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.26)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: right, reward: 2.11127124391
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 2.1112712439115873, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.11)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: 0.89594629165
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 0.8959462916497491, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 0.90)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 0.928082904202
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 0.928082904201518, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 0.93)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: forward, reward: 1.85204495386
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.8520449538646355, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.85)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: forward, reward: 1.30179510006
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': 1.3017951000587562, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent drove forward instead of left. (rewarded 1.30)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: -0.177990183217
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'left'), 'deadline': 17, 't': 13, 'action': None, 'reward': -0.17799018321725169, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.18)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: None, reward: -4.83186380261
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 16, 't': 14, 'action': None, 'reward': -4.831863802610506, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.83)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 1.22319961145
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.223199611448833, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 1.22)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: -9.63100974973
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': -9.631009749729616, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -9.63)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: 2.19291513825
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 2.1929151382544942, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.19)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: right, reward: 1.42363697876
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 1.423636978764262, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 1.42)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: 2.15697696825
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': 2.156976968245475, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.16)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 2.37726799417
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 20, 'action': 'forward', 'reward': 2.377267994170502, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.38)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 60
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (6, 4), deadline = 25
Simulating trial. . .
epsilon = 0.5488; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5488; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5488; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: forward, reward: 0.962485285111
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 0.9624852851113186, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent drove forward instead of right. (rewarded 0.96)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: 1.48201890188
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 1.4820189018768146, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent drove left instead of right. (rewarded 1.48)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: forward, reward: 1.3822307579
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'right'), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 1.3822307579002957, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'right')
Agent followed the waypoint forward. (rewarded 1.38)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 1.36280086893
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.3628008689251834, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.36)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 0.99518933853
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 0.9951893385304558, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.00)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: left, reward: 1.5896046481
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.589604648098947, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 1.59)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: right, reward: 2.2773134809
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.277313480902584, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 2.28)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: right, reward: 1.09263039339
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.0926303933879353, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.09)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: left, reward: 1.89411004189
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 1.8941100418894203, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.89)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: -9.287678565
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': -9.287678564995238, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.29)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 1.44768645305
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.4476864530499105, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.45)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: forward, reward: 0.936499842555
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 0.9364998425550948, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 0.94)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 0.822043996348
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 0.8220439963476038, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.82)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: left, reward: 2.60015816051
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 2.600158160513838, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.60)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: forward, reward: -39.4469085365
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': -39.44690853650747, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.45)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 1.43679835502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.4367983550235548, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent drove right instead of left. (rewarded 1.44)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 2.27227787958
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 2.272277879575518, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.27)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.77479202448
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 17, 'action': None, 'reward': 1.7747920244803495, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.77)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 0.849233283649
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 7, 't': 18, 'action': None, 'reward': 0.8492332836490601, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.85)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: -0.121576050605
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'left'), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': -0.12157605060496846, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'left')
Agent drove forward instead of left. (rewarded -0.12)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: -0.0555796130864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 5, 't': 20, 'action': 'forward', 'reward': -0.05557961308637349, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded -0.06)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: -10.1911222016
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': -10.191122201623427, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -10.19)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: -9.26258761381
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': -9.262587613807067, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.26)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.27585339804
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.2758533980427569, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.28)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.37366693916
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.3736669391617236, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.37)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 61
\-------------------------
Environment.reset(): Trial set up with start = (3, 2), destination = (2, 5), deadline = 20
Simulating trial. . .
epsilon = 0.5434; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5434; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5434; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: right, reward: 1.90511068538
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.9051106853773367, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.91)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: -4.51256616151
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': -4.512566161514813, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.51)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: -5.15468479736
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': -5.1546847973636325, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.15)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: -4.67476981164
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': -4.674769811635875, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.67)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: forward, reward: -9.28994674916
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': -9.289946749158208, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -9.29)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 2.60070877597
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.600708775967716, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.60)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 2.36960655774
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'right'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.369606557736282, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 2.37)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: -19.595041737
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': -19.595041736982502, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.60)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: 2.43887192051
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 2.4388719205111213, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 2.44)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 2.42960155389
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.429601553885291, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.43)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 1.61878301973
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.6187830197290327, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 1.62)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: -0.18398286479
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': -0.18398286479001036, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded -0.18)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: 1.62368005441
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 1.6236800544127261, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.62)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: -10.1952816965
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': -10.195281696478785, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent attempted driving left through a red light. (rewarded -10.20)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: -9.87781887241
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -9.877818872411426, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -9.88)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: -10.4887569165
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -10.488756916509203, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.49)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: 1.31152718478
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.3115271847838224, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 1.31)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: left, reward: 1.26030445647
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 1.2603044564684782, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 1.26)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: -10.6799561905
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': -10.679956190478268, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.68)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 1.41088421387
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 1.410884213870881, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.41)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 62
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (6, 6), deadline = 20
Simulating trial. . .
epsilon = 0.5379; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5379; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: 2.66775891191
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.667758911907341, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.67)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: left, reward: -9.03074142413
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -9.030741424130127, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -9.03)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: left, reward: -40.4617774712
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': -40.46177747115611, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.46)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: 1.25260476436
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.2526047643613762, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.25)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: 2.41475584086
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 2.414755840857731, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.41)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 1.52015891547
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.5201589154724533, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.52)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: -10.3719021308
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': -10.371902130841343, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent attempted driving left through a red light. (rewarded -10.37)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: right, reward: 0.740071255272
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.7400712552717146, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove right instead of left. (rewarded 0.74)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 1.79361007473
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.7936100747291372, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.79)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: forward, reward: 1.12596836997
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.125968369971312, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.13)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: left, reward: -9.4651842985
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -9.465184298502026, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.47)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: forward, reward: 2.55063611549
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 2.5506361154861494, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.55)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 63
\-------------------------
Environment.reset(): Trial set up with start = (4, 7), destination = (7, 5), deadline = 25
Simulating trial. . .
epsilon = 0.5326; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: -39.2655665526
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', 'right'), 'deadline': 25, 't': 0, 'action': 'left', 'reward': -39.26556655255527, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.27)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.47937902848
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.4793790284832786, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.48)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: -9.03085408885
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': -9.030854088853637, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.03)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: -39.0175068502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': -39.01750685018361, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.02)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 0.612221290451
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 0.6122212904507404, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent drove right instead of forward. (rewarded 0.61)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 2.92788774427
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.9278877442698743, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.93)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 2.87004192454
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.8700419245433784, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.87)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: 0.540765709762
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.5407657097620389, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.54)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 2.01753758653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.0175375865289946, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.02)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: right, reward: 0.199637272262
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'forward'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.19963727226206274, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 0.20)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: -4.38283305996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 15, 't': 10, 'action': None, 'reward': -4.382833059964586, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.38)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: 2.00279064257
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 2.002790642572226, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.00)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.80185298505
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.8018529850510883, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.80)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 1.55149465973
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 1.5514946597290673, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.55)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 1.66351811321
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 1.6635181132074082, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.66)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.28053436115
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.2805343611482167, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.28)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 0.798169987132
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 0.7981699871318753, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.80)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: -5.25784771991
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'right'), 'deadline': 8, 't': 17, 'action': None, 'reward': -5.257847719908833, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.26)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 1.20805143719
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': 1.2080514371916382, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded 1.21)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.10519612387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 6, 't': 19, 'action': None, 'reward': 2.105196123873605, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.11)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 0.63219247976
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', 'forward'), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.6321924797595067, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', 'forward')
Agent properly idled at a red light. (rewarded 0.63)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 1.464139119
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 1.4641391190045188, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.46)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 0.671335911722
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.6713359117220292, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 0.67)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: -19.3699375892
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', None), 'deadline': 2, 't': 23, 'action': 'right', 'reward': -19.369937589203477, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.37)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: -40.7337833996
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'right'), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': -40.733783399553516, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, 'forward', 'right')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.73)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 64
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (4, 6), deadline = 20
Simulating trial. . .
epsilon = 0.5273; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5273; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5273; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: 1.56586964498
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.565869644978106, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'left', 'left')
Agent drove left instead of right. (rewarded 1.57)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 2.25140552116
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.251405521163095, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.25)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: right, reward: 0.205460551758
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.20546055175763678, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.21)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 1.94010329923
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.940103299233576, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.94)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: 1.2583121558
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.2583121557951125, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent drove right instead of forward. (rewarded 1.26)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: forward, reward: -9.156917657
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': -9.1569176570003, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -9.16)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 1.64131265198
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.6413126519827388, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.64)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: forward, reward: 1.01952101164
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.01952101164064, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.02)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: left, reward: 1.3220407392
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.3220407391955422, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.32)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 1.25143125112
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.2514312511242214, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.25)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: 1.50839071799
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.508390717989486, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 1.51)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 65
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (3, 2), deadline = 30
Simulating trial. . .
epsilon = 0.5220; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 1.06506743132
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.065067431324539, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.07)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 0.570464548246
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 0.5704645482459855, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove right instead of forward. (rewarded 0.57)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: left, reward: 2.64048401888
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 28, 't': 2, 'action': 'left', 'reward': 2.6404840188758607, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.64)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: right, reward: -20.0945495449
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 27, 't': 3, 'action': 'right', 'reward': -20.094549544879765, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.09)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: -39.0332720133
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': -39.03327201329376, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.03)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 1.40139012009
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.401390120091533, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.40)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: -39.8284808487
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'right'), 'deadline': 24, 't': 6, 'action': 'left', 'reward': -39.82848084870794, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.83)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.77629119052
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.7762911905225407, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.78)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: 1.087298796
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 22, 't': 8, 'action': 'left', 'reward': 1.0872987960012854, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.09)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.62448767334
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 21, 't': 9, 'action': None, 'reward': 2.6244876733355453, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.62)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: -10.4650673692
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'right', 'forward'), 'deadline': 20, 't': 10, 'action': 'left', 'reward': -10.465067369220279, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'forward')
Agent attempted driving left through a red light. (rewarded -10.47)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.1232211884
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'forward'), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.123221188397305, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.12)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 1.33561264022
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.3356126402219934, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.34)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 2.38677686418
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 2.3867768641810096, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.39)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.12482606787
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.1248260678716457, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.12)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: 1.13454741126
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.1345474112586065, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.13)
47% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 66
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (3, 7), deadline = 30
Simulating trial. . .
epsilon = 0.5169; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: left, reward: 2.7105354461
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 2.7105354461040214, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.71)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: -4.95555366517
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', None), 'deadline': 29, 't': 1, 'action': None, 'reward': -4.955553665174309, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.96)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: right, reward: 0.0726986117572
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 0.07269861175717673, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 0.07)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: -9.28022039676
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': 'left', 'reward': -9.280220396764529, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.28)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: 2.12420582566
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 2.124205825656782, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.12)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 1.69315165146
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.6931516514625764, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.69)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: right, reward: 0.129441149515
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 0.12944114951499464, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.13)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.90583675755
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.9058367575456807, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.91)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.30942471029
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.309424710294133, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.31)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.9112514882
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.9112514881971787, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.91)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: right, reward: 0.878519477624
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 0.8785194776236738, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove right instead of left. (rewarded 0.88)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: 1.61574407408
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.6157440740778632, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 1.62)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 1.38351448166
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.3835144816644132, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.38)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 1.60105697066
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': None, 'reward': 1.601056970657763, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.60)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: 0.17835891944
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 0.17835891943965088, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.18)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: 0.770028704621
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 15, 'action': 'left', 'reward': 0.7700287046209369, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.77)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: 1.11932284484
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': 1.1193228448367132, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.12)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 0.649537941895
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 0.6495379418947154, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.65)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: forward, reward: -10.0687273726
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 12, 't': 18, 'action': 'forward', 'reward': -10.068727372610518, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -10.07)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: 0.895015337524
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 0.8950153375239501, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.90)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: left, reward: -9.47833955616
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 20, 'action': 'left', 'reward': -9.478339556156401, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.48)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: 0.584628739746
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 0.5846287397457718, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.58)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: forward, reward: -9.89319833429
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': -9.893198334288254, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.89)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 0.931371215807
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'forward'), 'deadline': 7, 't': 23, 'action': None, 'reward': 0.9313712158070766, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 0.93)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 1.56777864302
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 1.5677786430165084, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.57)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 0.948653260762
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 5, 't': 25, 'action': None, 'reward': 0.9486532607616998, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 0.95)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.31143985258
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 4, 't': 26, 'action': None, 'reward': 1.3114398525767137, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.31)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.60152883833
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 3, 't': 27, 'action': None, 'reward': 1.6015288383309418, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.60)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 0.793700624909
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 2, 't': 28, 'action': 'forward', 'reward': 0.7937006249086551, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent followed the waypoint forward. (rewarded 0.79)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 2.01194797909
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 1, 't': 29, 'action': 'right', 'reward': 2.011947979085182, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 2.01)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 67
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (4, 7), deadline = 25
Simulating trial. . .
epsilon = 0.5117; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5117; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5117; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5117; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 2.63260663814
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.6326066381416893, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.63)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 2.88180007379
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.8818000737891243, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.88)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 0.416379958088
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 0.416379958088259, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.42)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: -0.01357206898
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': -0.013572068980011842, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded -0.01)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: -5.5591782262
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 21, 't': 4, 'action': None, 'reward': -5.559178226204088, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.56)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 2.88166981379
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 2.8816698137942707, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.88)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 2.08167081289
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.081670812886837, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.08)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 1.00458479239
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.004584792390764, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.00)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 1.4307559834
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.4307559834029004, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent drove right instead of forward. (rewarded 1.43)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: -20.2912732775
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': -20.291273277485356, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.29)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 2.81789392798
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.8178939279767414, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.82)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 0.983416981924
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 0.9834169819240143, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 0.98)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.45537025059
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.455370250585947, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.46)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: -4.92924671839
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': -4.929246718392406, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.93)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 2.40391298888
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 2.403912988882888, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.40)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 0.86750462376
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 10, 't': 15, 'action': None, 'reward': 0.8675046237604274, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 0.87)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: -40.0338198537
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 9, 't': 16, 'action': 'left', 'reward': -40.033819853717105, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.03)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 1.41908045326
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.4190804532577022, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.42)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.75828304232
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.7582830423224183, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.76)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.41115876156
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 6, 't': 19, 'action': None, 'reward': 1.4111587615628118, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.41)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.68404116943
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.6840411694268185, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.68)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: -4.96428529307
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 4, 't': 21, 'action': None, 'reward': -4.964285293073525, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.96)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: -0.142463440889
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': -0.14246344088892227, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove right instead of forward. (rewarded -0.14)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: left, reward: 0.913775718011
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 2, 't': 23, 'action': 'left', 'reward': 0.9137757180112367, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent followed the waypoint left. (rewarded 0.91)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: -4.34107850305
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': -4.3410785030498396, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.34)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 68
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (1, 6), deadline = 20
Simulating trial. . .
epsilon = 0.5066; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5066; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5066; alpha = 0.5000
Simulating trial. . .
epsilon = 0.5066; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: left, reward: 0.936429992024
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 0.9364299920243853, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove left instead of right. (rewarded 0.94)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: 1.40903147349
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.4090314734889469, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.41)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: right, reward: 1.84279213173
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.842792131731601, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.84)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: left, reward: 1.64165546111
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 1.6416554611145604, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'left')
Agent followed the waypoint left. (rewarded 1.64)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: forward, reward: 1.5113268324
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.51132683239748, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.51)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: left, reward: -10.2890950462
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': -10.289095046172982, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.29)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 0.4455592519
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.44555925190008816, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent drove right instead of forward. (rewarded 0.45)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 1.09296167378
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.0929616737759387, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.09)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 1.56206486899
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.562064868989639, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.56)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 0.271559184293
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.27155918429339243, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.27)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 1.49822743282
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.4982274328155392, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.50)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: -4.18060575677
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 9, 't': 11, 'action': None, 'reward': -4.1806057567733514, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.18)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 2.39312893513
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 2.3931289351265277, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent followed the waypoint forward. (rewarded 2.39)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.82503181896
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.8250318189645955, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.83)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.32890006125
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.328900061252192, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.33)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: -4.85724461208
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 5, 't': 15, 'action': None, 'reward': -4.857244612075918, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.86)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 1.13528959607
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.1352895960711116, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent followed the waypoint forward. (rewarded 1.14)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: -10.1486244607
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -10.148624460670398, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.15)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 1.58108479226
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.5810847922635405, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.58)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: left, reward: 0.219211110349
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', 'left'), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 0.21921111034907226, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'left', 'right', 'left')
Agent drove left instead of right. (rewarded 0.22)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 69
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (6, 3), deadline = 30
Simulating trial. . .
epsilon = 0.5016; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 0.355755709304
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'forward'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 0.3557557093040207, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.36)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: 1.78217048871
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 1.7821704887082053, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent drove forward instead of left. (rewarded 1.78)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: 0.675689592446
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 0.6756895924458114, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', None)
Agent drove right instead of left. (rewarded 0.68)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 0.65815441285
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 0.6581544128497349, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent drove right instead of forward. (rewarded 0.66)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: left, reward: 1.06132015136
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 1.0613201513625397, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.06)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: left, reward: -10.0003385863
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': -10.000338586329354, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.00)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: right, reward: 1.82623325557
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 1.826233255569397, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 1.83)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 2.11476955254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 23, 't': 7, 'action': None, 'reward': 2.1147695525432546, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.11)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 2.30844482807
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.3084448280733287, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.31)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: right, reward: 0.141553907579
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'forward'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 0.14155390757885655, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'forward')
Agent drove right instead of left. (rewarded 0.14)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 1.75017593247
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.7501759324693151, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.75)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: left, reward: 0.335894233419
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 0.3358942334192542, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent drove left instead of right. (rewarded 0.34)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: left, reward: -19.8880598428
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'forward'), 'deadline': 18, 't': 12, 'action': 'left', 'reward': -19.888059842788127, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.89)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 2.76678804289
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 17, 't': 13, 'action': None, 'reward': 2.7667880428850715, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.77)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: -9.6292066555
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': -9.629206655504756, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.63)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.2005405873
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.2005405872983237, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.20)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: 0.967305466533
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 0.9673054665331515, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.97)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: left, reward: 1.60751445718
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 13, 't': 17, 'action': 'left', 'reward': 1.6075144571826796, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.61)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: left, reward: -10.0738805434
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 18, 'action': 'left', 'reward': -10.073880543364194, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.07)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: left, reward: 0.000539504084009
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 11, 't': 19, 'action': 'left', 'reward': 0.000539504084009379, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 0.00)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: -4.56449852307
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 10, 't': 20, 'action': None, 'reward': -4.5644985230733495, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.56)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: -19.4573614965
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 9, 't': 21, 'action': 'right', 'reward': -19.45736149648196, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.46)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: forward, reward: -10.98150208
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': -10.981502080002103, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.98)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: right, reward: 2.26357333526
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 2.2635733352627305, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.26)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: 1.61782795216
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 6, 't': 24, 'action': 'forward', 'reward': 1.617827952156136, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.62)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.31377419649
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 5, 't': 25, 'action': None, 'reward': 1.3137741964908012, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.31)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 1.28281112848
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 4, 't': 26, 'action': 'right', 'reward': 1.2828111284848585, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.28)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: -19.5031501749
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 3, 't': 27, 'action': 'right', 'reward': -19.503150174863325, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.50)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 1.02420454405
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 2, 't': 28, 'action': None, 'reward': 1.024204544053269, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.02)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 0.415134174825
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 1, 't': 29, 'action': 'right', 'reward': 0.4151341748248367, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.42)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 70
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (4, 7), deadline = 25
Simulating trial. . .
epsilon = 0.4966; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4966; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4966; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4966; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: -10.4215035675
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -10.421503567508204, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent attempted driving forward through a red light. (rewarded -10.42)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.15740143603
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.157401436030905, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.16)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.8696275057
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.8696275057029745, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.87)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.22668007102
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.2266800710198216, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.23)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 1.05351968045
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'forward'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.0535196804516467, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'forward')
Agent followed the waypoint right. (rewarded 1.05)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 0.953179366946
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.9531793669464725, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.95)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: -40.5554642922
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': -40.555464292179025, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.56)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.74309410177
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.7430941017746058, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.74)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: -10.936383726
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': -10.936383725951071, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.94)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.87835380779
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.8783538077904376, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.88)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.37861567988
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.3786156798778342, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.38)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: 1.93780994285
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.937809942854308, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 1.94)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: left, reward: 1.05148811377
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 13, 't': 12, 'action': 'left', 'reward': 1.0514881137741372, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.05)
48% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 71
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (4, 3), deadline = 30
Simulating trial. . .
epsilon = 0.4916; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4916; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4916; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 0.356810659593
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'forward'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 0.35681065959328895, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'forward')
Agent drove right instead of forward. (rewarded 0.36)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 0.377772507947
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 0.3777725079470077, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 0.38)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 1.32006773328
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 1.3200677332787685, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.32)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 0.749670402445
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', None), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 0.7496704024447193, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', None)
Agent drove right instead of forward. (rewarded 0.75)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: left, reward: 1.47697364146
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 1.4769736414642127, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 1.48)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 2.74906649226
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.7490664922597414, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.75)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: left, reward: 1.79263509058
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 1.7926350905760797, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.79)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: None, reward: -5.25994672754
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': -5.259946727538141, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.26)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: 1.18835408337
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 1.1883540833728161, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.19)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: None, reward: 1.40084239645
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.4008423964529368, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.40)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: None, reward: 0.9780842436
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 20, 't': 10, 'action': None, 'reward': 0.9780842435999959, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 0.98)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: 1.47050818224
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 1.4705081822411066, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.47)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: -5.84708153494
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 18, 't': 12, 'action': None, 'reward': -5.847081534938113, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.85)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: -9.64658822684
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': -9.646588226838917, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.65)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 0.934037245741
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 0.9340372457411599, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.93)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: forward, reward: -10.0725922604
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': -10.072592260364935, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.07)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.42330120569
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 14, 't': 16, 'action': None, 'reward': 1.4233012056910876, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.42)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.59422638907
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.5942263890746085, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.59)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 0.875917470108
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 0.8759174701082588, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.88)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: left, reward: 1.00793948275
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 19, 'action': 'left', 'reward': 1.007939482745192, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.01)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 1.24313098546
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 20, 'action': None, 'reward': 1.2431309854649877, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.24)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: 0.0965419336637
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 0.09654193366366226, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent drove forward instead of left. (rewarded 0.10)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 1.64301437583
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 8, 't': 22, 'action': None, 'reward': 1.6430143758287228, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.64)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: left, reward: -9.76322978767
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 7, 't': 23, 'action': 'left', 'reward': -9.763229787670497, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.76)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: left, reward: -10.1794458904
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': 'left', 'reward': -10.179445890419585, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.18)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 1.55955380165
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 25, 'action': None, 'reward': 1.5595538016549728, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.56)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: forward, reward: 0.431558160055
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'right'), 'deadline': 4, 't': 26, 'action': 'forward', 'reward': 0.4315581600550644, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'right')
Agent drove forward instead of left. (rewarded 0.43)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: left, reward: 2.22060434771
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 3, 't': 27, 'action': 'left', 'reward': 2.220604347714935, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.22)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 0.831247000714
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 2, 't': 28, 'action': None, 'reward': 0.8312470007140689, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 0.83)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 0.206416170413
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 1, 't': 29, 'action': None, 'reward': 0.2064161704125489, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.21)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 72
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (1, 3), deadline = 25
Simulating trial. . .
epsilon = 0.4868; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: right, reward: 1.92056535983
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.9205653598250776, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.92)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: 2.19817507325
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 2.198175073253336, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.20)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 0.556249156903
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 0.5562491569033858, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent drove right instead of left. (rewarded 0.56)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: forward, reward: -9.45532415544
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': -9.455324155436593, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.46)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 1.32390235207
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.323902352071205, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove right instead of forward. (rewarded 1.32)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: left, reward: 1.1674564591
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.1674564591005885, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.17)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 1.30812837909
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'forward'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.308128379085157, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'forward')
Agent drove right instead of left. (rewarded 1.31)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: left, reward: -10.5482436567
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 7, 'action': 'left', 'reward': -10.548243656655368, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.55)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: -9.29500115235
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -9.295001152351274, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.30)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 1.4272159052
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.427215905195288, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.43)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: left, reward: -9.60153557877
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': -9.601535578774385, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.60)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 1.89990126633
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.899901266328113, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.90)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 1.22050238373
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.2205023837309896, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.22)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: -5.95303132671
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': -5.953031326708382, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.95)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 2.41356253844
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 2.413562538444287, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.41)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.55598733125
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.5559873312515524, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.56)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.60863061346
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 2.6086306134646717, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.61)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 1.14075368381
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 1.1407536838089878, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.14)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 2.26123572845
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 2.2612357284500746, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 2.26)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: forward, reward: 1.6567662268
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 1.6567662267986676, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 1.66)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 73
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (5, 2), deadline = 25
Simulating trial. . .
epsilon = 0.4819; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: -4.51247618903
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': -4.51247618903152, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.51)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 0.729937297555
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.7299372975548435, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove right instead of left. (rewarded 0.73)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: -9.66449772751
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': -9.664497727513375, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.66)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 0.650228450193
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 0.6502284501929656, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.65)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 1.53761256234
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.5376125623402328, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.54)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 2.82248821882
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.8224882188181883, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.82)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 2.16036746483
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.16036746482923, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.16)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: -4.50637925737
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': -4.506379257367557, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.51)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 1.23389110856
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.233891108563046, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.23)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: -9.97730188257
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', 'forward'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': -9.977301882573817, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.98)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 0.394523605091
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'forward'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.3945236050914732, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.39)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.37718998709
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.3771899870917226, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 1.38)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 0.893531346043
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 0.8935313460431169, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.89)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 1.05241867104
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.052418671043385, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', None)
Agent followed the waypoint right. (rewarded 1.05)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: -10.0405267449
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -10.040526744876678, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', None)
Agent attempted driving left through a red light. (rewarded -10.04)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.56292256889
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.5629225688871689, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.56)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 0.896529839038
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 0.896529839038243, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 0.90)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 0.397925884445
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 0.3979258844451221, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.40)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: -0.154898073909
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'forward'), 'deadline': 7, 't': 18, 'action': 'right', 'reward': -0.154898073909024, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'forward')
Agent drove right instead of left. (rewarded -0.15)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 0.475933731756
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 19, 'action': None, 'reward': 0.47593373175573195, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.48)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 2.02767270909
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 2.0276727090893556, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.03)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 0.532647555013
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 0.5326475550132013, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 0.53)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 1.72914966964
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 3, 't': 22, 'action': None, 'reward': 1.7291496696432076, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.73)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 0.923514317123
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 2, 't': 23, 'action': None, 'reward': 0.9235143171232141, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.92)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: left, reward: -0.334108157118
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 1, 't': 24, 'action': 'left', 'reward': -0.3341081571179163, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded -0.33)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 74
\-------------------------
Environment.reset(): Trial set up with start = (8, 7), destination = (5, 3), deadline = 25
Simulating trial. . .
epsilon = 0.4771; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4771; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.93008652989
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.9300865298940024, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.93)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: -9.69000701386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': -9.690007013860633, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.69)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 1.15325874717
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.1532587471717761, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.15)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: forward, reward: 1.46399876575
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.4639987657500846, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 1.46)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 2.14406511699
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.144065116991651, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.14)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 1.27005986104
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.2700598610354459, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.27)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: forward, reward: 0.676408777864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 0.6764087778639961, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.68)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: forward, reward: 1.45434669432
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.4543466943174825, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent drove forward instead of left. (rewarded 1.45)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 2.82258083138
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.822580831384653, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.82)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: left, reward: 1.45590663426
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.4559066342552491, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.46)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: -10.6312862147
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': -10.63128621466001, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.63)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 0.922183329119
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': 0.9221833291190744, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.92)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 2.5929289352
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.5929289351963725, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.59)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: forward, reward: 2.43649340559
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 2.436493405593814, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.44)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: forward, reward: 1.70895786429
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 1.7089578642896444, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.71)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 75
\-------------------------
Environment.reset(): Trial set up with start = (7, 6), destination = (3, 2), deadline = 30
Simulating trial. . .
epsilon = 0.4724; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: 1.53320867568
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 1.5332086756813863, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 1.53)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 2.94931926804
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.949319268044042, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.95)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 2.33666736965
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.336667369648676, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.34)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: 0.394639538418
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 0.39463953841843147, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.39)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: -4.1539227296
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': -4.153922729601408, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.15)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: 1.97842013506
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 1.978420135064278, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.98)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 1.51160903171
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.5116090317070674, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.51)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: left, reward: 1.13745849698
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 1.1374584969802628, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent drove left instead of forward. (rewarded 1.14)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: left, reward: -10.5780636613
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 22, 't': 8, 'action': 'left', 'reward': -10.578063661321728, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent attempted driving left through a red light. (rewarded -10.58)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: right, reward: 2.77591526811
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 2.775915268110931, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.78)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 1.79897670411
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 1.7989767041065239, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.80)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: forward, reward: 0.729340187353
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 0.7293401873529649, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 0.73)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 0.884584080045
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', None), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 0.8845840800451303, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', None)
Agent followed the waypoint right. (rewarded 0.88)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 1.30922962222
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 1.309229622218147, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.31)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 1.44773657493
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.4477365749348927, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.45)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 1.01070461727
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.010704617272336, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent drove right instead of left. (rewarded 1.01)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 0.793134052062
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'left'), 'deadline': 14, 't': 16, 'action': None, 'reward': 0.7931340520622567, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.79)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.59425873767
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 13, 't': 17, 'action': None, 'reward': 2.59425873767042, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.59)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 1.65175765399
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 1.651757653986531, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.65)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: forward, reward: 1.52151220657
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': 1.5215122065655886, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 1.52)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 1.97293219717
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 1.9729321971747862, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.97)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: right, reward: 1.09763045507
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 1.0976304550656055, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 1.10)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: 0.536510108245
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': 0.5365101082454287, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.54)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: left, reward: 0.873387130761
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 23, 'action': 'left', 'reward': 0.8733871307614001, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.87)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 76
\-------------------------
Environment.reset(): Trial set up with start = (7, 4), destination = (2, 7), deadline = 30
Simulating trial. . .
epsilon = 0.4677; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4677; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 1.23463754001
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.2346375400079632, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.23)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: 1.16507531842
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'right', 'left'), 'deadline': 29, 't': 1, 'action': 'left', 'reward': 1.1650753184198042, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'right', 'left')
Agent followed the waypoint left. (rewarded 1.17)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.35849261223
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.3584926122281713, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.36)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: left, reward: 0.27860227822
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 27, 't': 3, 'action': 'left', 'reward': 0.2786022782204097, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.28)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: right, reward: 1.74508246555
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 1.745082465547438, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.75)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: forward, reward: -9.17628355986
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': -9.176283559864551, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -9.18)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.32991730353
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.3299173035289709, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.33)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: right, reward: 1.49512659026
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.4951265902628674, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.50)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: left, reward: 1.86685216171
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 22, 't': 8, 'action': 'left', 'reward': 1.86685216171401, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 1.87)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 2.25451992376
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 2.2545199237630156, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.25)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: -5.13649855417
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 20, 't': 10, 'action': None, 'reward': -5.136498554171899, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.14)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 1.24947457506
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.2494745750608658, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.25)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: left, reward: -10.893772455
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 12, 'action': 'left', 'reward': -10.893772454957325, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.89)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 0.521558973902
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': None, 'reward': 0.5215589739023632, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.52)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: -4.41702141619
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': -4.417021416187089, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.42)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 1.45994468581
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.4599446858144962, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.46)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 0.705530977005
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 0.7055309770051656, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent drove right instead of forward. (rewarded 0.71)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: 0.449351836527
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': 0.44935183652714517, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent drove forward instead of left. (rewarded 0.45)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 2.51772744844
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 18, 'action': None, 'reward': 2.5177274484449192, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.52)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: left, reward: 1.4332385631
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 19, 'action': 'left', 'reward': 1.433238563101502, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.43)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 0.656406642328
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 10, 't': 20, 'action': None, 'reward': 0.6564066423277732, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.66)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: 1.97308736047
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'left', 'reward': 1.973087360468446, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.97)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 0.908671349141
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 0.9086713491405065, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.91)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 0.477674951537
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 0.47767495153666595, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent drove right instead of left. (rewarded 0.48)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 2.20134911657
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 2.201349116572156, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.20)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: -20.0753687861
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'left', 'right', 'right'), 'deadline': 5, 't': 25, 'action': 'left', 'reward': -20.075368786074826, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.08)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 1.9769083032
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 4, 't': 26, 'action': 'right', 'reward': 1.97690830320199, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 1.98)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 0.607632028194
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 3, 't': 27, 'action': None, 'reward': 0.6076320281940648, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.61)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 0.309233608711
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 28, 'action': None, 'reward': 0.3092336087108092, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.31)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 2.028247082
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 1, 't': 29, 'action': 'forward', 'reward': 2.0282470820009397, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.03)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 77
\-------------------------
Environment.reset(): Trial set up with start = (5, 5), destination = (1, 5), deadline = 20
Simulating trial. . .
epsilon = 0.4630; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 1.12555269448
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.1255526944773382, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent drove forward instead of right. (rewarded 1.13)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: left, reward: -40.3016156881
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -40.30161568809238, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.30)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 0.840984605389
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.8409846053886626, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.84)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: 1.66999038053
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.6699903805313117, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 1.67)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: right, reward: 0.225169388926
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.22516938892555804, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.23)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 2.4658461887
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.4658461886988086, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.47)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: left, reward: 1.52508079215
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.5250807921463831, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 1.53)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: forward, reward: 2.14556155495
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.1455615549488005, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.15)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 1.55752834698
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.5575283469801733, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.56)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 2.69356196501
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 2.69356196500561, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.69)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.03718967853
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.0371896785309094, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.04)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 0.61145719596
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.6114571959601744, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove right instead of forward. (rewarded 0.61)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: forward, reward: -10.42802292
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': -10.428022920025537, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.43)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 1.38092178198
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.380921781976118, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.38)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: -9.11947146156
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -9.119471461564041, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent attempted driving left through a red light. (rewarded -9.12)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 1.06036345889
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.0603634588862307, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.06)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.84345418802
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.8434541880218305, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.84)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.09069995391
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.090699953912614, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.09)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: -10.0294103778
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': -10.02941037778548, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.03)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 0.885173937291
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 0.8851739372914151, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', 'right', None, None)
Agent followed the waypoint forward. (rewarded 0.89)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 78
\-------------------------
Environment.reset(): Trial set up with start = (5, 7), destination = (1, 2), deadline = 25
Simulating trial. . .
epsilon = 0.4584; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.17893338951
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.17893338951296, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.18)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.98130821479
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.981308214785291, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.98)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.41230812793
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.41230812793017, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.41)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: 2.32208412947
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 2.3220841294731427, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.32)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: 1.74661271042
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.746612710418326, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.75)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: forward, reward: 0.426025832485
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'right'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 0.42602583248508863, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'right')
Agent drove forward instead of left. (rewarded 0.43)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: 1.79728660249
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.7972866024903174, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.80)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 0.45392900738
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 0.45392900738028485, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.45)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: 2.51993515993
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 2.5199351599317454, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 2.52)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: left, reward: 0.710460712955
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 0.7104607129547584, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 0.71)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: forward, reward: 1.04692218774
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 1.0469221877350776, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 1.05)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: forward, reward: -39.3427439029
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': -39.34274390286478, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.34)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 1.32420580804
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.3242058080353247, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.32)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 0.671165309148
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 12, 't': 13, 'action': None, 'reward': 0.671165309148364, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.67)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: 0.370020256703
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'left'), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 0.3700202567030987, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'left')
Agent drove left instead of right. (rewarded 0.37)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: 2.05976576063
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'left'), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 2.059765760628536, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'left')
Agent followed the waypoint left. (rewarded 2.06)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 0.0295207917202
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 0.02952079172023836, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 0.03)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: -40.7147024285
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': -40.71470242847041, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.71)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: left, reward: -40.5397033184
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 7, 't': 18, 'action': 'left', 'reward': -40.53970331838341, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.54)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 1.56879224522
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 1.5687922452161578, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.57)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: forward, reward: -9.38367477427
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 5, 't': 20, 'action': 'forward', 'reward': -9.383674774266522, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -9.38)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: right, reward: 1.97407636158
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 1.9740763615845662, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.97)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 0.154665827799
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.15466582779940374, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.15)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 0.162471631939
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'left'), 'deadline': 2, 't': 23, 'action': 'right', 'reward': 0.16247163193941194, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'left')
Agent drove right instead of left. (rewarded 0.16)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 1.08374432818
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 1, 't': 24, 'action': 'right', 'reward': 1.083744328181267, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.08)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 79
\-------------------------
Environment.reset(): Trial set up with start = (5, 2), destination = (2, 7), deadline = 20
Simulating trial. . .
epsilon = 0.4538; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: -40.2832351619
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'forward'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -40.28323516189182, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.28)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 1.53427412637
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.534274126369574, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.53)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 2.52066420136
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.5206642013606118, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.52)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: left, reward: 2.74696746627
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 2.746967466271536, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.75)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 1.93310358076
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.9331035807561745, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.93)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: 2.25567759708
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.2556775970849237, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.26)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: left, reward: 0.769883264806
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 0.7698832648058633, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 0.77)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: 2.73962713033
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 2.739627130333718, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.74)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 1.91482794539
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.9148279453921184, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.91)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 2.78911115421
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 2.789111154205009, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.79)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 80
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (4, 5), deadline = 20
Simulating trial. . .
epsilon = 0.4493; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 1.96171127142
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.961711271419208, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.96)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 1.83417620205
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.8341762020506591, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.83)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: -4.96596172486
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'right'), 'deadline': 18, 't': 2, 'action': None, 'reward': -4.965961724855068, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.97)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: -39.1353036925
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': -39.13530369245756, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.14)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 1.12791406299
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.1279140629898068, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.13)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: 1.75033770786
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.750337707862923, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', None)
Agent followed the waypoint forward. (rewarded 1.75)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: -9.13736480087
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -9.137364800873991, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.14)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: forward, reward: 2.40354220654
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.4035422065435412, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.40)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: 2.27623357455
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 2.276233574548499, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.28)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: left, reward: -20.5718551454
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'right'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': -20.571855145439216, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.57)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: 0.930694504819
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'right'), 'deadline': 10, 't': 10, 'action': None, 'reward': 0.9306945048192281, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'right')
Agent properly idled at a red light. (rewarded 0.93)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 0.619894216552
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.6198942165515262, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent drove right instead of left. (rewarded 0.62)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: right, reward: 0.733202987026
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.733202987026329, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 0.73)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: right, reward: 2.1763216678
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 2.1763216677967074, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.18)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 0.732526332988
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.7325263329876599, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.73)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: right, reward: 1.43809116251
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.4380911625143582, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.44)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: 2.18207927895
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 2.1820792789530628, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.18)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: -10.6054351041
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -10.605435104060003, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -10.61)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: 1.61039269527
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.6103926952678105, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.61)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: 0.776217688642
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'forward'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 0.776217688641592, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'right', 'left', 'forward')
Agent drove forward instead of left. (rewarded 0.78)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 81
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (8, 5), deadline = 20
Simulating trial. . .
epsilon = 0.4449; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 2.70228953169
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.7022895316875446, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.70)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 1.0546648739
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.0546648739025812, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.05)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: -4.65697447421
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 18, 't': 2, 'action': None, 'reward': -4.656974474207984, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.66)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: left, reward: 0.761092684043
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 0.7610926840431956, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove left instead of forward. (rewarded 0.76)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 1.2161261449
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.2161261449025496, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.22)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.32925879511
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.3292587951118273, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.33)
70% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 82
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (7, 3), deadline = 20
Simulating trial. . .
epsilon = 0.4404; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4404; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4404; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4404; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: forward, reward: 0.0765170570019
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 0.0765170570019047, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.08)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 1.38226853696
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.3822685369638552, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.38)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 2.46993251046
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 2.469932510463822, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.47)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 1.21326881844
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'right'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.2132688184367164, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'right')
Agent drove right instead of forward. (rewarded 1.21)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: 1.06481608503
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.0648160850296935, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove forward instead of left. (rewarded 1.06)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: 1.67691027509
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.676910275093516, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.68)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 2.18937432537
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 2.1893743253729836, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.19)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: left, reward: 0.922876182612
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 0.9228761826120371, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.92)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 83
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (7, 5), deadline = 30
Simulating trial. . .
epsilon = 0.4360; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4360; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4360; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4360; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 1.17230510218
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.172305102180069, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 1.17)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: 1.21656054804
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 1.216560548041871, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.22)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: left, reward: 1.0191475132
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 28, 't': 2, 'action': 'left', 'reward': 1.019147513196378, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove left instead of forward. (rewarded 1.02)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: None, reward: 1.58630559647
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.5863055964742652, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.59)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: None, reward: 2.82724888164
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.8272488816385364, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.83)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: right, reward: 1.6555773997
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.6555773996988357, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.66)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.845260347
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.8452603470043998, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.85)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 1.70554720548
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.7055472054838865, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.71)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: forward, reward: -10.4243387563
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': -10.424338756299639, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.42)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.91425146378
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.9142514637812063, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.91)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: right, reward: 1.72420273453
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.7242027345345308, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.72)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: -39.1338422961
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 19, 't': 11, 'action': 'left', 'reward': -39.13384229606235, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.13)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: right, reward: 1.36652476069
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 1.3665247606942588, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent drove right instead of left. (rewarded 1.37)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: forward, reward: 1.59158172642
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.5915817264178391, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.59)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: 0.894749219251
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 16, 't': 14, 'action': 'left', 'reward': 0.8947492192512423, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent drove left instead of right. (rewarded 0.89)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: left, reward: 0.000360980275088
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 15, 'action': 'left', 'reward': 0.00036098027508801867, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 0.00)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: forward, reward: 1.26913602105
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': 1.269136021045889, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.27)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: forward, reward: -0.222095961587
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': -0.2220959615871203, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded -0.22)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 1.3888337765
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 1.3888337764991139, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.39)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: -9.42487132038
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': -9.424871320382312, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.42)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 0.8442054636
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 10, 't': 20, 'action': None, 'reward': 0.844205463599526, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 0.84)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 2.13333985951
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 2.1333398595054165, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.13)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 0.513228249969
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': 0.5132282499688083, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 0.51)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.43288917551
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 7, 't': 23, 'action': None, 'reward': 2.4328891755094024, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.43)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.00573562789
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 6, 't': 24, 'action': None, 'reward': 2.005735627888745, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.01)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.36852752595
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 5, 't': 25, 'action': None, 'reward': 1.3685275259521754, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.37)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 1.98677710356
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 4, 't': 26, 'action': 'forward', 'reward': 1.9867771035572077, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent followed the waypoint forward. (rewarded 1.99)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: -0.0171975041252
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 27, 'action': 'right', 'reward': -0.017197504125217478, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded -0.02)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: left, reward: 1.72514901495
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 28, 'action': 'left', 'reward': 1.725149014953563, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.73)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 0.170535946136
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 1, 't': 29, 'action': None, 'reward': 0.17053594613600898, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.17)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 84
\-------------------------
Environment.reset(): Trial set up with start = (8, 7), destination = (4, 5), deadline = 30
Simulating trial. . .
epsilon = 0.4317; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 2.67965270233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.679652702325688, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.68)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 2.58678635822
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.5867863582233737, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.59)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.44533586454
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.4453358645356815, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.45)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 2.81477025927
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.814770259265341, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.81)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.10606961616
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.1060696161630363, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.11)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.83807661426
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.8380766142610743, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.84)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: forward, reward: 0.790096990925
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 0.7900969909254095, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 0.79)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 1.38421049951
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.3842104995126638, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.38)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: forward, reward: -10.1000859954
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': -10.100085995399956, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.10)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: left, reward: 2.38624938626
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 2.386249386260829, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.39)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: -4.00688335292
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': -4.006883352921913, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.01)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 0.614236525137
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 0.6142365251367882, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 0.61)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 0.927243226398
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 0.9272432263979877, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.93)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: -4.2584542666
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 17, 't': 13, 'action': None, 'reward': -4.258454266603485, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.26)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: forward, reward: -0.197999789491
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': -0.19799978949108787, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove forward instead of left. (rewarded -0.20)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 1.37074094215
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.3707409421465864, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.37)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 0.831122889391
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 0.8311228893906364, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 0.83)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 1.49889834436
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.498898344357333, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.50)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: left, reward: -39.9632059931
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', 'forward'), 'deadline': 12, 't': 18, 'action': 'left', 'reward': -39.963205993104864, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.96)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: -0.332833228797
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': -0.3328332287970134, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove forward instead of left. (rewarded -0.33)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: -10.7093181814
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 10, 't': 20, 'action': 'forward', 'reward': -10.709318181395336, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.71)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: 0.7410637577
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 0.741063757699927, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.74)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 1.92371662775
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 22, 'action': None, 'reward': 1.9237166277526696, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.92)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: -0.536452834447
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'right'), 'deadline': 7, 't': 23, 'action': 'right', 'reward': -0.5364528344469565, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'right')
Agent drove right instead of forward. (rewarded -0.54)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: 1.16539171186
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': None, 'reward': 1.165391711857078, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.17)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: 1.54279377797
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 5, 't': 25, 'action': 'left', 'reward': 1.542793777973836, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.54)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: -9.9087987957
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 4, 't': 26, 'action': 'left', 'reward': -9.908798795698798, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.91)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 0.672754388352
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 3, 't': 27, 'action': None, 'reward': 0.6727543883522045, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 0.67)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: 1.12460026283
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 2, 't': 28, 'action': 'forward', 'reward': 1.1246002628322977, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.12)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: -5.62355805649
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 1, 't': 29, 'action': None, 'reward': -5.623558056489632, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.62)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 85
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (4, 4), deadline = 25
Simulating trial. . .
epsilon = 0.4274; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4274; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4274; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: -40.6876931367
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'left'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -40.68769313672642, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.69)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.82814792855
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.8281479285525313, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.83)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.6942048674
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.694204867404713, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.69)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: -9.18875367252
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'left', 'reward': -9.188753672518823, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.19)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.7251915597
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.7251915597025436, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.73)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 1.64816392865
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.6481639286501208, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.65)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: -4.83317908185
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': -4.833179081850799, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.83)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 2.75531549597
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.755315495974717, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.76)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: left, reward: -10.1194020997
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': -10.119402099723912, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.12)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 2.23374967223
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.2337496722346417, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.23)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: -0.141434946757
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': -0.14143494675738666, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove left instead of forward. (rewarded -0.14)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: forward, reward: 0.188696282059
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'right', None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 0.18869628205877198, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'right', None)
Agent drove forward instead of right. (rewarded 0.19)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: right, reward: 1.1385511278
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.1385511278038978, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.14)
48% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 86
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (1, 3), deadline = 25
Simulating trial. . .
epsilon = 0.4232; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4232; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4232; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: right, reward: 2.14752749023
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.1475274902282404, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.15)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: 2.63924525549
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 2.6392452554897643, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent followed the waypoint forward. (rewarded 2.64)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 1.44441597569
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.4444159756885229, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove right instead of left. (rewarded 1.44)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 2.60519145899
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.6051914589928526, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.61)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: left, reward: 1.62482821266
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 1.6248282126606788, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent drove left instead of forward. (rewarded 1.62)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 2.31508694111
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 2.3150869411127513, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.32)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: None, reward: 1.59952757398
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.5995275739799526, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.60)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: right, reward: 0.956911634425
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.9569116344254758, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.96)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: forward, reward: 1.40533347131
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 1.4053334713076864, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded 1.41)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 0.279599555245
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.279599555245039, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.28)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: forward, reward: -9.17648503157
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': -9.176485031571593, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -9.18)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: 1.97696645883
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.9769664588312839, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.98)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 1.60773881839
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.607738818394727, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.61)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 1.14635469081
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.1463546908146072, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.15)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 0.481714968034
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', 'forward'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 0.4817149680342675, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', 'forward')
Agent drove right instead of forward. (rewarded 0.48)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 2.57360671913
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.5736067191309395, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.57)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: None, reward: 1.9177545378
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.9177545378014735, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.92)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: left, reward: -20.4230502646
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 8, 't': 17, 'action': 'left', 'reward': -20.423050264648385, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.42)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: left, reward: 1.91441930819
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 7, 't': 18, 'action': 'left', 'reward': 1.9144193081892844, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.91)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: forward, reward: 0.930943517444
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 0.9309435174437203, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove forward instead of left. (rewarded 0.93)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: 2.21968235751
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 5, 't': 20, 'action': None, 'reward': 2.2196823575124203, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 2.22)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: 1.0339730356
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 1.033973035601118, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove right instead of left. (rewarded 1.03)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 1.90901089169
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 1.9090108916854214, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.91)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 1.63827830619
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.6382783061932897, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.64)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: left, reward: -39.4504249486
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 1, 't': 24, 'action': 'left', 'reward': -39.45042494863879, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.45)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 87
\-------------------------
Environment.reset(): Trial set up with start = (2, 7), destination = (8, 5), deadline = 20
Simulating trial. . .
epsilon = 0.4190; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 1.28496881903
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.2849688190290658, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', None)
Agent drove right instead of left. (rewarded 1.28)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 1.61731296767
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'right'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.6173129676717342, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.62)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: 2.18931385387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'right'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.1893138538711643, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'right')
Agent followed the waypoint right. (rewarded 2.19)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: 1.58228725288
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.5822872528813277, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent followed the waypoint forward. (rewarded 1.58)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 2.34428930965
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.3442893096494286, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.34)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: 0.794933808995
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 0.7949338089947826, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent drove left instead of forward. (rewarded 0.79)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 1.45117286073
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.451172860732764, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.45)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: forward, reward: 0.291500875221
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 0.2915008752211665, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', 'left')
Agent drove forward instead of right. (rewarded 0.29)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 2.58564630218
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.585646302175048, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.59)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 88
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (4, 2), deadline = 20
Simulating trial. . .
epsilon = 0.4148; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 1.45889586579
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.4588958657942666, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'forward')
Agent drove right instead of left. (rewarded 1.46)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: right, reward: 1.13703828738
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.1370382873825926, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.14)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: left, reward: -9.68132743498
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': -9.681327434980147, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.68)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: left, reward: -9.66311071588
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -9.663110715880107, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.66)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: left, reward: 1.40451923491
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.404519234912969, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.40)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: forward, reward: 2.53445357507
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 2.5344535750707715, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.53)
70% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 89
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (1, 7), deadline = 20
Simulating trial. . .
epsilon = 0.4107; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: left, reward: -19.8900752675
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'forward'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -19.89007526749962, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.89)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: 2.62107537074
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.6210753707395362, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 2.62)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 2.3317925909
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.33179259090426, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.33)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 2.64523121275
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.6452312127512783, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.65)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: -9.22803907134
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': -9.228039071344117, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.23)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 0.611058074137
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.6110580741373919, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.61)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: -10.8684544497
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -10.868454449686297, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.87)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: -9.48438120979
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', 'right', 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -9.48438120979474, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'right', 'left')
Agent attempted driving forward through a red light. (rewarded -9.48)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 1.29271599951
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.2927159995086943, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 1.29)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: 1.9205153385
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.9205153384953528, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.92)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: 2.67107917211
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 2.67107917210872, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.67)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 1.74470920639
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.7447092063852685, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.74)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 0.806987618937
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.8069876189373639, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 0.81)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: forward, reward: 1.93080186356
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.9308018635625817, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.93)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 1.2928417292
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.2928417291968786, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.29)
25% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 90
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (2, 6), deadline = 20
Simulating trial. . .
epsilon = 0.4066; alpha = 0.5000
Simulating trial. . .
epsilon = 0.4066; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 2.50244656636
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.5024465663608684, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.50)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.2077354349
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.2077354349000609, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.21)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 2.63476261029
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.634762610289925, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.63)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.33794921101
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.3379492110070443, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.34)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.0441338194
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.0441338194040097, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.04)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 1.17700879328
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.177008793279107, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.18)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 2.56458209792
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.5645820979201615, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.56)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 2.68380829416
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.683808294158826, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.68)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 2.37224241665
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.372242416650345, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.37)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 1.47349927418
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.4734992741832944, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.47)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.65235692142
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.6523569214198055, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.65)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 1.38802213704
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.388022137037915, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.39)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 2.28790135534
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 2.28790135534489, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.29)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 0.686987178114
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 7, 't': 13, 'action': None, 'reward': 0.6869871781136825, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.69)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: -0.0140619489789
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': -0.014061948978896122, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded -0.01)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 1.71621184038
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.7162118403783906, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.72)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: forward, reward: -9.28660538235
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': -9.286605382349066, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.29)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: -0.619868777368
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 3, 't': 17, 'action': 'right', 'reward': -0.6198687773681072, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded -0.62)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 0.868525996053
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 0.868525996052741, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.87)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 1.28724458596
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 1.2872445859612665, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.29)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 91
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (3, 5), deadline = 20
Simulating trial. . .
epsilon = 0.4025; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: forward, reward: 1.80111215651
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.8011121565096393, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove forward instead of left. (rewarded 1.80)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: None, reward: -5.31781172616
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', 'right'), 'deadline': 19, 't': 1, 'action': None, 'reward': -5.3178117261617555, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.32)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: left, reward: 1.7590893945
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.7590893944967316, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.76)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 2.36856079915
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.3685607991510516, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.37)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: forward, reward: 0.992013166531
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 0.9920131665309604, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent followed the waypoint forward. (rewarded 0.99)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 2.53113749604
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.5311374960398276, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.53)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: right, reward: 0.951588702077
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'right'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.9515887020771385, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'right')
Agent followed the waypoint right. (rewarded 0.95)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 1.74252055086
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.742520550864494, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.74)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 1.98686432816
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.986864328161252, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.99)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 2.17489390109
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.174893901088409, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.17)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: forward, reward: 1.56176053947
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.561760539466456, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.56)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: -4.16626958272
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': -4.166269582722573, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.17)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: forward, reward: 2.49731157539
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'right'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 2.497311575385046, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'right')
Agent followed the waypoint forward. (rewarded 2.50)
35% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 92
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (2, 6), deadline = 20
Simulating trial. . .
epsilon = 0.3985; alpha = 0.5000
Simulating trial. . .
epsilon = 0.3985; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 2.45389841033
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.4538984103298347, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.45)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 2.71393553474
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.7139355347415837, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.71)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: -4.97629062637
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 18, 't': 2, 'action': None, 'reward': -4.976290626372034, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.98)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 2.48886198483
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.4888619848308116, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.49)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.80467662294
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.8046766229382767, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.80)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 1.06209668037
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'right'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.0620966803675442, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'right')
Agent drove right instead of forward. (rewarded 1.06)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: 1.5815263573
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.5815263573039213, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.58)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: left, reward: 0.615154283699
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 0.6151542836994668, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.62)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 1.98086772255
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.9808677225504485, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.98)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 2.42008872594
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 2.420088725937611, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.42)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 93
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (6, 5), deadline = 20
Simulating trial. . .
epsilon = 0.3946; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: 1.60330323456
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.6033032345578635, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.60)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: 0.360258977927
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 0.3602589779273696, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove left instead of right. (rewarded 0.36)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: right, reward: 0.720660705059
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.7206607050588519, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.72)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 1.53309626058
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.5330962605810399, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove right instead of left. (rewarded 1.53)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 1.66146556701
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.6614655670075904, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.66)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: -19.0428055301
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': -19.042805530098292, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.04)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 1.4996873953
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.499687395296104, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.50)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: left, reward: -40.7037942201
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': -40.70379422007135, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.70)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 1.56975170099
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.569751700985234, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.57)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 2.40303338191
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 2.403033381906722, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.40)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.22404628054
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.224046280541626, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.22)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: left, reward: -9.77832966548
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': -9.77832966548313, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.78)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 1.70526441867
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.7052644186747365, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.71)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 1.11353162247
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.1135316224725942, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent followed the waypoint forward. (rewarded 1.11)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 2.26639265191
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 2.266392651905615, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.27)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 0.108646900695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 0.10864690069493821, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent drove right instead of forward. (rewarded 0.11)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: -0.191368342012
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': -0.19136834201217234, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded -0.19)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: -0.0871118971908
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': None, 'reward': -0.08711189719084278, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.09)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 2.14624025882
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 2.146240258823261, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 2.15)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 1.33201517846
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 1.3320151784623822, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.33)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 94
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (8, 4), deadline = 30
Simulating trial. . .
epsilon = 0.3906; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: left, reward: -9.76744679145
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', 'forward'), 'deadline': 30, 't': 0, 'action': 'left', 'reward': -9.767446791451079, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -9.77)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 1.88668980836
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.8866898083603858, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.89)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.79457294165
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.7945729416510825, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.79)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.91389992474
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.9138999247368322, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.91)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 1.73126018949
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 1.7312601894888981, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent drove right instead of left. (rewarded 1.73)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: left, reward: 2.65603349757
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 2.656033497570095, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.66)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 1.32715111279
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.3271511127904885, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.33)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 1.11421412287
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.1142141228657847, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.11)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 2.70892092101
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.7089209210086382, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.71)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 2.84168503963
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 21, 't': 9, 'action': None, 'reward': 2.8416850396306703, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.84)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 1.73368666552
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.733686665520346, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.73)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: left, reward: 2.16017003944
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 2.1601700394393637, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.16)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 0.390800662146
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 0.39080066214584297, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.39)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: right, reward: 0.888204557155
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 0.888204557154634, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 0.89)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 0.9431106186
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 0.9431106186002967, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.94)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 2.07372134404
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 15, 't': 15, 'action': None, 'reward': 2.073721344039417, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.07)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: right, reward: 1.18401874285
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 1.1840187428492936, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.18)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 1.12265797122
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.1226579712189453, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.12)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: forward, reward: -9.10300781984
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 12, 't': 18, 'action': 'forward', 'reward': -9.103007819842436, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.10)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: forward, reward: -10.1111373938
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': -10.111137393753877, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.11)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: 2.1333957223
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 2.133395722295351, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.13)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: 2.45344685703
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 2.4534468570325965, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent followed the waypoint forward. (rewarded 2.45)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: -10.7765387617
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': -10.776538761747641, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.78)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 2.03069519044
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 7, 't': 23, 'action': 'forward', 'reward': 2.0306951904363615, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent followed the waypoint forward. (rewarded 2.03)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: left, reward: -9.13091942009
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': 'left', 'reward': -9.130919420089828, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.13)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 1.92294573835
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 5, 't': 25, 'action': 'forward', 'reward': 1.922945738350228, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.92)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 0.842370560215
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 4, 't': 26, 'action': 'right', 'reward': 0.842370560215236, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.84)
10% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 95
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (8, 6), deadline = 30
Simulating trial. . .
epsilon = 0.3867; alpha = 0.5000
Simulating trial. . .
epsilon = 0.3867; alpha = 0.5000
Simulating trial. . .
epsilon = 0.3867; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: -40.0853915764
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': -40.0853915764336, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.09)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 2.61352136504
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 2.613521365041988, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.61)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 1.63412522457
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 1.634125224570392, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.63)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: 1.2083962838
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 1.208396283797609, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent drove forward instead of left. (rewarded 1.21)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: forward, reward: 0.199807801544
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 0.19980780154428224, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent drove forward instead of left. (rewarded 0.20)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: None, reward: 2.23204400872
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.232044008724044, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.23)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: left, reward: -10.5865004248
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'left'), 'deadline': 24, 't': 6, 'action': 'left', 'reward': -10.58650042476341, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'left')
Agent attempted driving left through a red light. (rewarded -10.59)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: 2.47331621883
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 2.4733162188323083, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.47)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: left, reward: 0.294825104752
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': 0.2948251047515359, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 0.29)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 2.7547565688
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 2.754756568795102, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.75)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 0.279736087138
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'right'), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 0.2797360871375274, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'right')
Agent drove forward instead of right. (rewarded 0.28)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 1.36453106762
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 1.3645310676224454, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.36)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 1.40016559327
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.4001655932650487, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.40)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 1.93835061467
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 1.9383506146692695, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 1.94)
53% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 96
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (5, 2), deadline = 20
Simulating trial. . .
epsilon = 0.3829; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: 1.22946908022
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.22946908022464, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.23)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 2.64457896498
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.6445789649789955, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.64)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 0.670980782503
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.6709807825027766, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.67)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: forward, reward: 2.62926306795
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.6292630679472397, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.63)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: forward, reward: 1.47203473352
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.47203473352115, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.47)
75% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 97
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (2, 3), deadline = 25
Simulating trial. . .
epsilon = 0.3791; alpha = 0.5000
Simulating trial. . .
epsilon = 0.3791; alpha = 0.5000
Simulating trial. . .
epsilon = 0.3791; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: -5.52706044436
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': -5.5270604443634435, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.53)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: 0.471159804965
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 0.47115980496549237, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent drove left instead of right. (rewarded 0.47)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.03206275762
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.0320627576194745, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.03)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.50452688431
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.5045268843086834, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.50)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: 0.379726109835
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 0.3797261098349508, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 0.38)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.80538022347
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.8053802234716023, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.81)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.81499618165
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.8149961816508386, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 2.81)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: right, reward: 1.42752486324
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.4275248632437345, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.43)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: left, reward: -20.9041789534
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'forward'), 'deadline': 17, 't': 8, 'action': 'left', 'reward': -20.90417895338636, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.90)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 2.47857648015
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.47857648015141, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.48)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 2.07510108603
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.075101086031309, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.08)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 1.06522767606
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.065227676057575, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.07)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 1.46640174036
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.466401740363278, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.47)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: left, reward: 2.06162952905
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 2.0616295290491706, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.06)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: None, reward: 1.68662321205
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.6866232120488245, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.69)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: 1.01200699056
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 1.0120069905609486, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.01)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.81892969085
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.8189296908465973, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.82)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: left, reward: 1.1497793686
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 1.1497793686030873, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.15)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: -4.16863087428
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 7, 't': 18, 'action': None, 'reward': -4.168630874275637, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.17)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 1.66558241044
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 1.6655824104424823, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.67)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 0.654289935866
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.6542899358661762, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.65)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: -0.307952568694
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': -0.30795256869396825, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded -0.31)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 1.92162310863
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 3, 't': 22, 'action': None, 'reward': 1.9216231086306659, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.92)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: -0.401381117133
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 2, 't': 23, 'action': None, 'reward': -0.4013811171325763, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.40)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: -0.491587078955
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'left'), 'deadline': 1, 't': 24, 'action': None, 'reward': -0.49158707895471787, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'left', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.49)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 98
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (4, 3), deadline = 30
Simulating trial. . .
epsilon = 0.3753; alpha = 0.5000
Simulating trial. . .
epsilon = 0.3753; alpha = 0.5000
Simulating trial. . .
epsilon = 0.3753; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.98873021299
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'left'), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.9887302129902589, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.99)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: -10.6262334243
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': 'left', 'reward': -10.626233424273192, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.63)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: -39.9725199293
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': -39.97251992929046, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.97)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.65806027209
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.6580602720907351, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.66)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.03438886331
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.034388863309533, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.03)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 1.87249503785
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.8724950378509857, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.87)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: left, reward: 1.1595823109
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 1.15958231090011, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.16)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: forward, reward: 0.257304169423
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', 'forward'), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 0.25730416942252454, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', 'forward')
Agent drove forward instead of right. (rewarded 0.26)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: forward, reward: -9.05448031072
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'right'), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': -9.054480310723273, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'right')
Agent attempted driving forward through a red light. (rewarded -9.05)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 2.26259106284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 2.262591062841959, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.26)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: left, reward: -40.0799522044
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'right'), 'deadline': 20, 't': 10, 'action': 'left', 'reward': -40.07995220437983, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.08)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 0.252115408017
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', None), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 0.25211540801727017, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', None)
Agent drove right instead of forward. (rewarded 0.25)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: left, reward: 1.76634858569
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 18, 't': 12, 'action': 'left', 'reward': 1.7663485856873107, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.77)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: left, reward: 1.0456542075
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 1.0456542075044384, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.05)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 1.78219833185
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.782198331851625, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.78)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: 0.418842726472
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 0.41884272647151366, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.42)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: left, reward: -10.2577674418
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'right', 'forward'), 'deadline': 14, 't': 16, 'action': 'left', 'reward': -10.257767441795398, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', 'forward')
Agent attempted driving left through a red light. (rewarded -10.26)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: left, reward: -9.21978289817
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 13, 't': 17, 'action': 'left', 'reward': -9.21978289817351, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.22)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 2.60311890997
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 12, 't': 18, 'action': None, 'reward': 2.6031189099677006, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.60)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: forward, reward: 0.582180318025
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': 0.5821803180253705, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.58)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: right, reward: 0.416373448394
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', None), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 0.41637344839414125, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', None)
Agent drove right instead of left. (rewarded 0.42)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 1.63804502436
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 1.6380450243582025, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.64)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 1.45794981033
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 8, 't': 22, 'action': None, 'reward': 1.457949810332014, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.46)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 0.996491777244
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 0.99649177724362, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent drove right instead of forward. (rewarded 1.00)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: left, reward: 2.12288889373
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 6, 't': 24, 'action': 'left', 'reward': 2.1228888937250785, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.12)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 1.97679141107
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 1.976791411069747, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.98)
13% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 99
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (1, 3), deadline = 20
Simulating trial. . .
epsilon = 0.3716; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 2.18207495034
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.1820749503360037, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.18)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: right, reward: 1.0135583345
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.0135583345036094, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.01)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 1.61487003424
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.6148700342445164, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 1.61)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: left, reward: -10.6822531793
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -10.682253179278002, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.68)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 2.82400557989
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.8240055798866637, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.82)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: left, reward: -0.036337614756
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': -0.03633761475598951, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded -0.04)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 1.37001688538
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.370016885381238, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.37)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: -5.46553780806
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': -5.465537808056938, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.47)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: left, reward: 1.16980218465
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.1698021846526023, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.17)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: left, reward: 1.78954697763
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.7895469776348085, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 1.79)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 0.851734983696
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'forward'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 0.8517349836960231, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'forward')
Agent followed the waypoint forward. (rewarded 0.85)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 2.37067173224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.3706717322405613, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.37)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 0.665513141781
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.6655131417810853, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.67)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: right, reward: -0.137182666291
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': -0.13718266629132214, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove right instead of left. (rewarded -0.14)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: 2.19977242145
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'right', 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.1997724214490457, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.20)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: 1.60295399525
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.6029539952476388, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.60)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: right, reward: 1.42338365444
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'forward'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.4233836544387248, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 1.42)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: right, reward: 1.34244344984
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 1.3424434498434707, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.34)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: -4.32616264383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 2, 't': 18, 'action': None, 'reward': -4.326162643833124, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.33)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: -4.57519187409
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 1, 't': 19, 'action': None, 'reward': -4.5751918740900095, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.58)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 100
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (6, 2), deadline = 20
Simulating trial. . .
epsilon = 0.3679; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: -20.2724186894
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': -20.27241868939142, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.27)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 1.21437787886
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.2143778788613466, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent drove right instead of left. (rewarded 1.21)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.02807716566
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.028077165658814, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.03)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: -9.71272365199
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': -9.712723651993773, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.71)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.96052612389
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.9605261238868374, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.96)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 0.614446800343
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.6144468003433343, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.61)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: 2.16745413506
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.167454135056116, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent followed the waypoint left. (rewarded 2.17)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: 1.68801610956
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.6880161095633637, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.69)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 0.64561231831
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.6456123183097512, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove right instead of forward. (rewarded 0.65)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 1.00668775047
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.0066877504730773, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.01)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: -20.3933122894
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -20.393312289402058, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.39)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: 1.22828473772
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'right', None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.2282847377160138, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'right', None)
Agent drove forward instead of right. (rewarded 1.23)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 1.93441336761
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.9344133676069502, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.93)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: 0.984358581138
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.9843585811376737, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 0.98)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 0.78438934456
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.7843893445603587, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 0.78)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: -9.78410269129
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'right'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': -9.78410269129214, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'right')
Agent attempted driving forward through a red light. (rewarded -9.78)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 0.907192570926
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 0.9071925709255764, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.91)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: forward, reward: 0.536807463245
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.5368074632453177, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.54)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: left, reward: -0.539327055528
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': -0.5393270555284317, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove left instead of forward. (rewarded -0.54)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: right, reward: 0.556062652168
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.5560626521684049, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 0.56)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 101
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (8, 6), deadline = 25
Simulating trial. . .
epsilon = 0.3642; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: left, reward: 0.220819568337
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 0.22081956833678507, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove left instead of forward. (rewarded 0.22)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 2.83640374225
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.8364037422481188, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.84)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: forward, reward: -9.26787240802
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': -9.267872408021523, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.27)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: -9.2887036098
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 22, 't': 3, 'action': 'left', 'reward': -9.2887036097961, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent attempted driving left through a red light. (rewarded -9.29)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: right, reward: 0.613167675723
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 0.6131676757227341, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.61)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: left, reward: -19.7370566456
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, 'left', 'right'), 'deadline': 20, 't': 5, 'action': 'left', 'reward': -19.737056645603573, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.74)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: left, reward: 1.95627998843
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.9562799884310456, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.96)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 1.35851288177
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.3585128817728709, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.36)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 2.15275655483
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.152756554834376, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.15)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: forward, reward: 1.093507966
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.093507965997656, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.09)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.1226690757
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.1226690757035078, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', None)
Agent followed the waypoint right. (rewarded 1.12)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: 1.69024951852
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.6902495185178243, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove left instead of forward. (rewarded 1.69)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: 0.540660609882
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 0.5406606098817466, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent drove forward instead of right. (rewarded 0.54)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 0.992436386205
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 0.9924363862054779, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.99)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 1.7539102139
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 1.7539102138997946, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.75)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: 1.07705686646
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 1.0770568664562366, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.08)
36% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 102
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (3, 2), deadline = 25
Simulating trial. . .
epsilon = 0.3606; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: left, reward: -40.4130223704
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'left'), 'deadline': 25, 't': 0, 'action': 'left', 'reward': -40.41302237037996, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.41)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 1.76111963916
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.7611196391576818, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.76)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 1.11700949279
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.1170094927920478, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.12)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: left, reward: -10.1278113274
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': -10.127811327396527, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -10.13)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: left, reward: -10.0878107125
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': -10.08781071251663, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -10.09)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 1.95832539104
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.9583253910429088, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.96)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: -0.06300892517
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': -0.06300892516998957, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded -0.06)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: -4.58761659076
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 18, 't': 7, 'action': None, 'reward': -4.587616590756689, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.59)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: 1.13908756382
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 1.1390875638165143, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.14)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.68317673558
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.683176735583815, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.68)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.27086234123
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.2708623412258577, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.27)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: -39.5483468778
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': -39.548346877814375, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.55)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.29471098352
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.294710983515239, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.29)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: left, reward: 2.04966888853
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 2.04966888853383, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.05)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: 1.47402261239
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 1.4740226123930225, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.47)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 103
\-------------------------
Environment.reset(): Trial set up with start = (3, 4), destination = (8, 6), deadline = 25
Simulating trial. . .
epsilon = 0.3570; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 1.15377438213
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.1537743821315782, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.15)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 1.41538502809
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.4153850280854048, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.42)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 2.88930576136
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.889305761360051, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.89)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: left, reward: -10.53700696
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 22, 't': 3, 'action': 'left', 'reward': -10.537006960031327, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.54)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 2.88474081258
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.8847408125835683, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.88)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: forward, reward: 0.700254806316
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 0.7002548063157185, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.70)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: -40.2308905479
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': -40.230890547859275, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.23)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: -20.2185780342
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': -20.218578034206946, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.22)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: 2.84196142683
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.8419614268299442, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.84)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: -0.0894075345287
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': -0.08940753452872308, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded -0.09)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 1.72440697944
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.7244069794354937, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 1.72)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: -20.865474536
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': -20.865474535988334, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.87)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 2.52748754773
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 2.52748754773282, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.53)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: -40.5040986785
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': -40.504098678535506, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.50)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: -9.2131261207
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': -9.21312612069759, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.21)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: -9.27824630515
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': 'left', 'reward': -9.278246305149343, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.28)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 0.280547544263
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 0.2805475442633377, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.28)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: 0.454363470518
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 0.4543634705180588, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.45)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: forward, reward: 0.319575390261
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': 0.3195753902613998, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.32)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: 0.583889090445
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'forward'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 0.5838890904448218, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'forward')
Agent drove right instead of left. (rewarded 0.58)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.25004363925
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 1.2500436392526932, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.25)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: 0.485864353358
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 0.4858643533582455, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent drove forward instead of right. (rewarded 0.49)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: 1.9289784405
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 1.9289784405009482, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.93)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 0.120540764841
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 2, 't': 23, 'action': 'right', 'reward': 0.1205407648411827, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.12)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.03793175725
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.0379317572510336, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.04)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 104
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (5, 7), deadline = 25
Simulating trial. . .
epsilon = 0.3535; alpha = 0.5000
Simulating trial. . .
epsilon = 0.3535; alpha = 0.5000
Simulating trial. . .
epsilon = 0.3535; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.71054144914
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.7105414491401831, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.71)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 1.75271363189
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.7527136318880776, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.75)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: -4.38587276481
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': -4.385872764809973, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.39)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: 2.4252964834
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.4252964834048107, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.43)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 2.25766386981
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.257663869814539, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.26)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 2.00165245817
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.00165245817299, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.00)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 1.70402696688
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.7040269668778296, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.70)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 0.525897388315
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.5258973883153151, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 0.53)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: 1.39078580674
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.3907858067425343, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.39)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 1.13090742266
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.130907422660015, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.13)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: left, reward: 1.312085464
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 1.3120854640021158, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.31)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 1.28237133305
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.2823713330470659, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove right instead of forward. (rewarded 1.28)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 1.13520694727
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.1352069472729354, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.14)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: -19.1921545362
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 12, 't': 13, 'action': 'left', 'reward': -19.192154536208836, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.19)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: -0.216497796956
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': -0.21649779695609472, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove forward instead of left. (rewarded -0.22)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: -0.0702168766741
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': -0.07021687667411713, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove forward instead of left. (rewarded -0.07)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: -10.5939314434
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': -10.593931443448191, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.59)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 1.53166434564
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 1.5316643456425494, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.53)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 0.675182504513
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 0.6751825045134523, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 0.68)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: -4.35559552574
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'right', None), 'deadline': 6, 't': 19, 'action': None, 'reward': -4.355595525737494, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.36)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: 0.978448972734
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 5, 't': 20, 'action': 'left', 'reward': 0.9784489727344108, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 0.98)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 0.469465518165
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 0.469465518165339, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent drove right instead of left. (rewarded 0.47)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.50184942559
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': 1.5018494255886592, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.50)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: left, reward: -20.2501002473
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 2, 't': 23, 'action': 'left', 'reward': -20.250100247341983, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.25)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: 1.04186993585
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 1, 't': 24, 'action': 'left', 'reward': 1.041869935853708, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 1.04)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 105
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (2, 5), deadline = 20
Simulating trial. . .
epsilon = 0.3499; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: left, reward: 1.72119226303
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.721192263027003, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 1.72)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: right, reward: 1.284539992
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.284539992002312, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.28)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.55948479571
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.5594847957058686, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.56)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: -39.1181239487
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -39.1181239486672, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.12)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: -10.2413827957
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': -10.241382795694195, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.24)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: 1.10928240442
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.1092824044220086, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.11)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 2.33225116062
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'right'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 2.332251160624306, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'right')
Agent followed the waypoint forward. (rewarded 2.33)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: left, reward: 1.85952611211
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.859526112110293, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove left instead of right. (rewarded 1.86)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: 2.27448054945
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.2744805494455296, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.27)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: left, reward: -10.6493849098
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': -10.649384909800263, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent attempted driving left through a red light. (rewarded -10.65)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: left, reward: -9.95619893494
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': -9.956198934941867, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.96)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 1.08677965575
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.086779655750975, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.09)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: forward, reward: -39.4354612531
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': -39.435461253112564, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.44)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: 0.940280227243
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.9402802272433831, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 0.94)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 0.932498024883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.9324980248832881, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.93)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.70556114353
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.7055611435285518, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.71)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 1.6368036058
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.6368036057964974, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.64)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 0.714495957435
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', 'right'), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.7144959574351251, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', 'right')
Agent followed the waypoint right. (rewarded 0.71)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.26693199006
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.2669319900649358, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.27)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 0.56530656111
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.5653065611102162, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.57)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 106
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (3, 5), deadline = 20
Simulating trial. . .
epsilon = 0.3465; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 0.837588522409
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.8375885224089374, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 0.84)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: 2.29973372168
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 2.2997337216792086, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.30)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: -39.9930500056
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 18, 't': 2, 'action': 'left', 'reward': -39.99305000556782, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.99)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 2.00158899755
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.0015889975470573, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.00)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 2.80692491933
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.8069249193344765, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.81)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.15500197682
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.1550019768161508, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', 'forward')
Agent drove right instead of forward. (rewarded 1.16)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 1.98055520017
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.9805552001662947, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.98)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 0.484617155017
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.4846171550167848, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', None)
Agent drove right instead of left. (rewarded 0.48)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 1.17759782181
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.177597821812938, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 1.18)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: forward, reward: 1.72087960736
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.7208796073577957, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.72)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 2.20236773575
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 2.202367735746762, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.20)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 1.01208681504
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.0120868150398545, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.01)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 1.36985726055
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.369857260554342, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.37)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: 1.83223591183
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.8322359118289429, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.83)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.95119595893
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.9511959589337682, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.95)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: -0.185021813132
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': -0.18502181313220833, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded -0.19)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 0.598992840678
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.5989928406781297, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.60)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.30649849811
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.3064984981129324, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.31)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 1.70161128578
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 1.7016112857820196, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.70)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.33912904748
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.3391290474764548, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.34)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 107
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (5, 2), deadline = 25
Simulating trial. . .
epsilon = 0.3430; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 1.83773058062
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.8377305806193012, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.84)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 2.22603976951
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.2260397695053817, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.23)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 2.34148940495
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.341489404947162, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.34)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: -4.86699460047
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': -4.8669946004716005, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.87)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: left, reward: 1.15489557535
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 1.1548955753487349, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.15)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: -9.81079757013
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': -9.810797570127983, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.81)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 0.717463425783
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.7174634257829063, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.72)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 1.46364178136
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.463641781357516, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove right instead of left. (rewarded 1.46)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.85059254794
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.8505925479355323, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.85)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: 0.281131275581
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 0.2811312755814226, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 0.28)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: -9.31816697203
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': -9.318166972031273, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.32)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: 1.85642400612
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.856424006123472, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 1.86)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 1.87959563938
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 1.8795956393770366, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.88)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: -5.1352232014
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 12, 't': 13, 'action': None, 'reward': -5.135223201403123, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.14)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: -0.251431302362
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -0.2514313023624327, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent drove left instead of forward. (rewarded -0.25)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 1.16349609587
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'right'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.1634960958724498, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 1.16)
36% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 108
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (1, 6), deadline = 30
Simulating trial. . .
epsilon = 0.3396; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 2.7430489712
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.743048971198829, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.74)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 1.51527299574
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.515272995741143, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.52)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 2.1509523609
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.1509523608993266, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.15)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: right, reward: 0.930376159336
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 0.930376159336446, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.93)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 0.147812506341
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', None), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 0.14781250634148935, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', None)
Agent drove right instead of forward. (rewarded 0.15)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: right, reward: 0.144080235897
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 0.14408023589739882, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent drove right instead of left. (rewarded 0.14)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: right, reward: 1.541875317
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 1.5418753169978392, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove right instead of forward. (rewarded 1.54)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 1.49520742755
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.4952074275481462, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.50)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 0.998610456079
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 22, 't': 8, 'action': None, 'reward': 0.9986104560794236, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.00)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 1.18281119095
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.1828111909464076, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.18)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: forward, reward: 1.71266641388
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 1.7126664138828886, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.71)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 1.0649958736
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.0649958735983165, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.06)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 2.25938079246
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.2593807924602443, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.26)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 0.878415660327
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 13, 'action': None, 'reward': 0.8784156603265141, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.88)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: right, reward: 0.224115233292
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 0.22411523329220606, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 0.22)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 1.52806523293
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.5280652329343392, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.53)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: 1.73810812149
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': 1.7381081214887686, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.74)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: 1.54055834017
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': 1.540558340169497, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.54)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.15965799881
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 12, 't': 18, 'action': None, 'reward': 2.1596579988061775, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.16)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 1.11224911406
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 19, 'action': 'forward', 'reward': 1.1122491140602504, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.11)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: -20.1948887558
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', 'right'), 'deadline': 10, 't': 20, 'action': 'right', 'reward': -20.19488875578119, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', 'right')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.19)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.226463142
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 9, 't': 21, 'action': None, 'reward': 2.2264631419963807, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.23)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 1.04193161739
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 1.0419316173860278, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 1.04)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 1.25090909558
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 7, 't': 23, 'action': None, 'reward': 1.2509090955769868, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.25)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 1.42024269805
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 1.420242698047054, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.42)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: 0.587157137174
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 5, 't': 25, 'action': 'forward', 'reward': 0.587157137174001, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 0.59)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 1.58483856106
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 4, 't': 26, 'action': 'right', 'reward': 1.5848385610592004, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.58)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 1.3995117799
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 3, 't': 27, 'action': None, 'reward': 1.3995117798996313, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.40)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 1.16931686875
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 2, 't': 28, 'action': 'right', 'reward': 1.1693168687538114, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.17)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.16681126467
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 1, 't': 29, 'action': None, 'reward': 1.1668112646671975, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.17)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 109
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (5, 3), deadline = 35
Simulating trial. . .
epsilon = 0.3362; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: -4.34053405877
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'right'), 'deadline': 35, 't': 0, 'action': None, 'reward': -4.340534058772456, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.34)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 1.05939303315
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 34, 't': 1, 'action': 'right', 'reward': 1.0593930331481092, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 1.06)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.21631889962
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 33, 't': 2, 'action': None, 'reward': 1.2163188996171452, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.22)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 2.64135606773
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 32, 't': 3, 'action': None, 'reward': 2.6413560677324277, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.64)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 1.13461082775
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 31, 't': 4, 'action': 'forward', 'reward': 1.1346108277476266, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.13)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 1.72869651156
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 30, 't': 5, 'action': 'forward', 'reward': 1.7286965115594881, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent followed the waypoint forward. (rewarded 1.73)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 1.30465809833
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 29, 't': 6, 'action': None, 'reward': 1.3046580983261675, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.30)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: left, reward: 1.35825129899
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 28, 't': 7, 'action': 'left', 'reward': 1.3582512989899396, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 1.36)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 1.01255309169
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 27, 't': 8, 'action': 'right', 'reward': 1.0125530916870045, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.01)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: 1.35049288153
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 26, 't': 9, 'action': 'left', 'reward': 1.3504928815319244, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.35)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: 1.58810477912
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'right'), 'deadline': 25, 't': 10, 'action': None, 'reward': 1.588104779122112, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'right')
Agent properly idled at a red light. (rewarded 1.59)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: -4.48950845023
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'right'), 'deadline': 24, 't': 11, 'action': None, 'reward': -4.48950845022633, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.49)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: right, reward: 0.690828228989
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 23, 't': 12, 'action': 'right', 'reward': 0.6908282289888817, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.69)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: left, reward: 0.926695669848
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 22, 't': 13, 'action': 'left', 'reward': 0.9266956698477762, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.93)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 1.59673300742
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 21, 't': 14, 'action': None, 'reward': 1.5967330074164856, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.60)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 1.88670252235
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 20, 't': 15, 'action': None, 'reward': 1.8867025223490224, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.89)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: left, reward: 2.2040200225
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 19, 't': 16, 'action': 'left', 'reward': 2.204020022496458, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.20)
51% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 110
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (7, 5), deadline = 30
Simulating trial. . .
epsilon = 0.3329; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: -5.74459691456
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': -5.744596914559013, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.74)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 2.38187978486
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 2.3818797848580564, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.38)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 2.34535775981
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 2.345357759807311, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.35)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.16276305598
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.1627630559814324, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.16)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 1.07730306079
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 1.0773030607904825, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.08)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 1.83726065933
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.837260659326585, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.84)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 2.75419771281
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 2.7541977128133546, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.75)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.55134264527
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.551342645269426, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.55)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 2.36146018964
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.3614601896421696, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.36)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: left, reward: 1.42263051695
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 1.4226305169466746, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.42)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 0.960177709125
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 0.9601777091248036, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.96)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: left, reward: -0.106226184614
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': -0.10622618461386535, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded -0.11)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: -20.2301785491
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': -20.230178549093843, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.23)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: -5.45513088352
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 17, 't': 13, 'action': None, 'reward': -5.455130883522865, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.46)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 1.85651229586
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.8565122958638376, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.86)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.04942436462
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'forward'), 'deadline': 15, 't': 15, 'action': None, 'reward': 2.049424364616809, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.05)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 2.57259443969
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 2.572594439691544, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.57)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: -0.0378603698839
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'right'), 'deadline': 13, 't': 17, 'action': 'right', 'reward': -0.03786036988385022, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'right')
Agent drove right instead of forward. (rewarded -0.04)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.68939450253
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.6893945025314683, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.69)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 0.635859308974
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 11, 't': 19, 'action': None, 'reward': 0.6358593089742735, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.64)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 0.681636482651
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 10, 't': 20, 'action': None, 'reward': 0.6816364826510504, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 0.68)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: 1.19558875568
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'left', 'reward': 1.1955887556765836, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.20)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: -19.8637648191
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 8, 't': 22, 'action': 'left', 'reward': -19.863764819126814, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.86)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: 1.13119839433
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 7, 't': 23, 'action': 'left', 'reward': 1.1311983943328028, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.13)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 1.66556845147
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 1.6655684514736422, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.67)
17% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 111
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (5, 3), deadline = 20
Simulating trial. . .
epsilon = 0.3296; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 1.94294062606
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.9429406260625903, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.94)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 2.48565669797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.48565669796733, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.49)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 2.35723897707
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.3572389770745583, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.36)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: forward, reward: 1.17870228329
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.1787022832930427, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.18)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: right, reward: 2.68583049558
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.685830495577343, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.69)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: left, reward: 1.9557121379
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.9557121379049696, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.96)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: left, reward: 0.423856010082
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 0.4238560100817521, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove left instead of forward. (rewarded 0.42)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 1.57225068816
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.5722506881607432, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.57)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: right, reward: 2.52096168857
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.520961688571278, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.52)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 112
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (4, 7), deadline = 20
Simulating trial. . .
epsilon = 0.3263; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 1.08869987827
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.0886998782698318, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.09)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: left, reward: 1.59680839779
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.5968083977896956, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.60)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 1.60332857177
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.6033285717724997, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.60)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 2.54989895134
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.549898951341846, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.55)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 0.244068408993
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.24406840899304727, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.24)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: 0.695932414476
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 0.6959324144763821, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded 0.70)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: left, reward: -10.3104376234
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': -10.310437623362947, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.31)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: -5.29806136316
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': -5.298061363158611, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.30)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: left, reward: 2.72848605131
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 2.728486051313022, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.73)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: left, reward: -9.69926891607
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'forward'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': -9.699268916073512, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -9.70)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: 0.434145002024
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'forward'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 0.4341450020238581, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'forward')
Agent drove forward instead of left. (rewarded 0.43)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: -19.817931065
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', 'left'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': -19.817931065031264, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.82)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 1.53825880511
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.5382588051106363, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.54)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: left, reward: 0.924251133368
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.9242511333680137, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.92)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 1.66866184887
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.6686618488703964, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.67)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: forward, reward: 0.103969063799
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 0.10396906379927517, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.10)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: left, reward: 2.31514167356
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 2.3151416735623807, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.32)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: 1.18721818535
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 1.1872181853539832, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.19)
10% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 113
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (4, 6), deadline = 30
Simulating trial. . .
epsilon = 0.3230; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 1.22078180132
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.2207818013185838, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.22)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 1.06525576524
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.0652557652421561, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.07)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 1.35074996311
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 1.3507499631050535, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.35)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: 1.65594542202
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.6559454220152314, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.66)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: -10.3193669272
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': -10.319366927233775, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.32)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: forward, reward: 0.27580477294
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 0.27580477294000805, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.28)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 1.6999969592
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 1.6999969592017945, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.70)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.27222376039
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'right'), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.2722237603891873, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 1.27)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 1.27487077389
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 1.2748707738912515, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 1.27)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 1.52472315289
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 1.5247231528911485, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.52)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 1.76318343469
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 1.7631834346865576, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent followed the waypoint forward. (rewarded 1.76)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: -39.3968151434
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': -39.396815143449174, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.40)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 0.35263280947
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 0.35263280946962394, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.35)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: 2.34813848818
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 17, 't': 13, 'action': None, 'reward': 2.348138488179513, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.35)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: left, reward: -39.2621703157
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'right'), 'deadline': 16, 't': 14, 'action': 'left', 'reward': -39.262170315683285, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.26)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: 2.45352436411
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 15, 't': 15, 'action': None, 'reward': 2.4535243641051423, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.45)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: 2.4014089433
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 14, 't': 16, 'action': None, 'reward': 2.40140894329723, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.40)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: left, reward: 1.96568858937
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 13, 't': 17, 'action': 'left', 'reward': 1.9656885893735798, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 1.97)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 0.299575974548
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'left'), 'deadline': 12, 't': 18, 'action': None, 'reward': 0.2995759745479565, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.30)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 2.36369019165
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 11, 't': 19, 'action': None, 'reward': 2.3636901916518838, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.36)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: right, reward: 2.55811883105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 2.558118831048484, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 2.56)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: forward, reward: 0.664391588675
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 0.6643915886748826, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.66)
27% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 114
\-------------------------
Environment.reset(): Trial set up with start = (7, 4), destination = (8, 7), deadline = 20
Simulating trial. . .
epsilon = 0.3198; alpha = 0.5000
Simulating trial. . .
epsilon = 0.3198; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 2.66492376734
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.6649237673392596, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.66)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 2.83091698545
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.8309169854476104, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.83)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 2.01748313745
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.017483137449182, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 2.02)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: forward, reward: 0.634300831693
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 0.6343008316933739, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.63)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 1.61772275491
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.6177227549052717, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.62)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: right, reward: 0.367808138159
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.36780813815879587, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove right instead of left. (rewarded 0.37)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 2.64838398559
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.6483839855914812, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.65)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: -39.064826341
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -39.064826341022794, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.06)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 1.29892307059
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.2989230705919161, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.30)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 1.23845249133
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.2384524913300756, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.24)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 1.14935431879
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.149354318790568, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.15)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.98609466725
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.9860946672549131, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.99)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: -9.16524318018
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -9.165243180183547, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent attempted driving left through a red light. (rewarded -9.17)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: 2.37951180143
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 2.379511801431186, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.38)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 115
\-------------------------
Environment.reset(): Trial set up with start = (5, 5), destination = (2, 4), deadline = 20
Simulating trial. . .
epsilon = 0.3166; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 0.73858690134
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.73858690134049, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', 'left')
Agent drove right instead of forward. (rewarded 0.74)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: 0.555639025866
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.5556390258655332, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent drove right instead of left. (rewarded 0.56)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: -5.37637837682
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 18, 't': 2, 'action': None, 'reward': -5.376378376818222, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.38)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.48952958974
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.4895295897376684, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.49)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 2.35230024038
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.3523002403776005, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.35)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: right, reward: 1.03870714359
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.038707143594025, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.04)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 0.119464150047
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.11946415004678723, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', None)
Agent drove right instead of left. (rewarded 0.12)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 1.17769401225
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.1776940122531943, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.18)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 2.06809821769
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.068098217693991, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.07)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 2.48103083475
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.4810308347477115, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.48)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: 0.475119135083
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.4751191350834475, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent drove right instead of left. (rewarded 0.48)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: -10.6687468012
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': -10.668746801183469, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -10.67)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 2.57711524275
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.577115242751991, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.58)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 2.16934797793
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 2.1693479779291787, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.17)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: -10.4758820855
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': -10.475882085544058, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.48)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 1.42493698406
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.4249369840581743, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 1.42)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: 1.04052240526
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 1.0405224052636062, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.04)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 0.342326810811
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.34232681081099703, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 0.34)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: 1.05128424429
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': 1.0512842442919752, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.05)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: -39.193512717
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -39.19351271704875, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.19)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 116
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (1, 6), deadline = 25
Simulating trial. . .
epsilon = 0.3135; alpha = 0.5000
Simulating trial. . .
epsilon = 0.3135; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: 1.62761134251
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.6276113425084955, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.63)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: 2.51553251864
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.5155325186425714, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.52)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: -9.78671916606
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': -9.786719166056304, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.79)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: 1.70904871322
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.7090487132173233, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.71)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: right, reward: -0.0253582118311
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': -0.02535821183111986, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded -0.03)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: right, reward: 1.17054784861
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.170547848606212, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.17)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: -39.1050852777
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'right'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -39.10508527773752, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'right')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.11)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 1.1576825484
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.157682548395294, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.16)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 1.61295250123
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.6129525012295884, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.61)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: right, reward: -20.9353454531
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': -20.935345453067896, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.94)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 2.26838808312
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.268388083119035, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.27)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: -5.01416856817
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': -5.014168568172655, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.01)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: 1.34757320853
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 1.3475732085272143, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.35)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 1.12239908267
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.1223990826678998, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.12)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: -10.4265102447
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -10.426510244690416, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.43)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: -0.22911323101
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': -0.22911323101027325, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded -0.23)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: 0.770224982727
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 0.7702249827273727, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove forward instead of left. (rewarded 0.77)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: 1.15421702379
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 1.1542170237928715, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent drove forward instead of left. (rewarded 1.15)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: -9.071999245
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': -9.071999245003047, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.07)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 0.98340500933
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 0.9834050093304421, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.98)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: forward, reward: 1.29674259607
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 5, 't': 20, 'action': 'forward', 'reward': 1.2967425960663417, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent followed the waypoint forward. (rewarded 1.30)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: 2.13493273131
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 2.134932731312264, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.13)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.68325031616
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 3, 't': 22, 'action': None, 'reward': 1.6832503161567405, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.68)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.297849866
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'right'), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.2978498659953113, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'right')
Agent properly idled at a red light. (rewarded 1.30)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 0.543578602445
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.5435786024449363, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.54)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 117
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (4, 3), deadline = 25
Simulating trial. . .
epsilon = 0.3104; alpha = 0.5000
Simulating trial. . .
epsilon = 0.3104; alpha = 0.5000
Simulating trial. . .
epsilon = 0.3104; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: -4.99619625134
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': -4.996196251343681, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.00)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 2.30214522335
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 2.3021452233524613, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.30)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.75288097072
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.752880970722811, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.75)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.70979282813
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.7097928281348398, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.71)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.1498632223
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.1498632222987195, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.15)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 1.3998059765
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.3998059764999842, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.40)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 2.81201119215
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.812011192147549, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.81)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: forward, reward: 1.32359935962
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.3235993596203552, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.32)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: forward, reward: -9.99443006646
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -9.994430066456262, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.99)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 0.957834729286
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 0.9578347292855474, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.96)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: forward, reward: -10.5196343229
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': -10.519634322924832, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.52)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 1.58902032961
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.589020329610626, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent drove right instead of left. (rewarded 1.59)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 1.81062640002
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.8106264000225576, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.81)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: -5.25684120998
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': -5.25684120998216, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.26)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: -4.30641067219
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': -4.3064106721856374, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.31)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 1.36061572149
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.360615721486474, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.36)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: -4.18482220098
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': -4.184822200981097, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.18)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 1.75373512124
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.753735121242028, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.75)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: -5.38535481113
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 7, 't': 18, 'action': None, 'reward': -5.38535481113084, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.39)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.44074049625
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 6, 't': 19, 'action': None, 'reward': 1.4407404962516264, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.44)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: -0.449238544813
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 5, 't': 20, 'action': 'right', 'reward': -0.4492385448134839, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded -0.45)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: left, reward: 1.81762718616
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 1.8176271861577917, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.82)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: 1.64833620701
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': 1.6483362070064356, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.65)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: 1.31889938834
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 2, 't': 23, 'action': 'forward', 'reward': 1.3188993883419708, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.32)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 1.06320877213
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.0632087721316419, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.06)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 118
\-------------------------
Environment.reset(): Trial set up with start = (7, 2), destination = (8, 5), deadline = 20
Simulating trial. . .
epsilon = 0.3073; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: left, reward: 0.432185067992
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 0.43218506799203116, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent drove left instead of forward. (rewarded 0.43)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 2.29283947522
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.292839475220792, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.29)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: -9.65772583922
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': -9.657725839217358, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.66)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.52357145071
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.5235714507143907, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.52)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 2.41377886042
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.4137788604190655, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.41)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: -39.8469619111
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': -39.846961911070935, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.85)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: 2.67075068296
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.6707506829648775, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.67)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 2.2058002987
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.205800298695368, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.21)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 1.5412443445
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.5412443444954376, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.54)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 1.93723161098
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.937231610981315, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.94)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: forward, reward: 1.6777340704
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.6777340704014252, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.68)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 119
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (2, 7), deadline = 25
Simulating trial. . .
epsilon = 0.3042; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.2743949554
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.274394955401639, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.27)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 2.59018052075
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.590180520753907, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.59)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.87031871825
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'right'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.8703187182536547, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.87)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.72859483906
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.7285948390558934, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.73)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.06930664987
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.0693066498665107, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.07)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: 1.78706257495
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.787062574946295, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.79)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 1.78803028291
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'right'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.7880302829125547, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'right')
Agent drove right instead of forward. (rewarded 1.79)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: 1.29964802651
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 1.299648026510789, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.30)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 2.81552448913
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 2.8155244891294307, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 2.82)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 2.18745968441
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.18745968440676, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.19)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 2.62901706009
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.629017060094995, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.63)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: -5.50460587899
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'right'), 'deadline': 14, 't': 11, 'action': None, 'reward': -5.5046058789943295, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.50)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 1.14933614847
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.1493361484704931, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.15)
48% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 120
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (7, 6), deadline = 25
Simulating trial. . .
epsilon = 0.3012; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.40278402798
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.402784027977962, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.40)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.34582174696
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.3458217469649774, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.35)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: 0.0285036441532
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 0.028503644153184293, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 0.03)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 1.26788327937
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.2678832793695647, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.27)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 1.68854639245
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.688546392446716, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.69)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 2.90859477534
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.9085947753430466, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.91)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 1.12197723574
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.1219772357413234, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.12)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 1.7497526236
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.7497526235990453, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.75)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: -10.0513361525
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -10.051336152503605, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -10.05)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: forward, reward: 1.0149553417
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.0149553417002508, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.01)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: left, reward: 0.162018529439
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 0.16201852943877137, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove left instead of forward. (rewarded 0.16)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 2.44328511577
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 2.4432851157708573, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.44)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 1.16595408261
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.165954082611348, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.17)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 0.791829912912
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 12, 't': 13, 'action': None, 'reward': 0.7918299129124111, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 0.79)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 0.924150441048
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 11, 't': 14, 'action': None, 'reward': 0.924150441047608, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.92)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 0.237087412846
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 0.23708741284554313, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.24)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.91129510833
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.9112951083292467, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.91)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 0.90985331137
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 8, 't': 17, 'action': None, 'reward': 0.9098533113696605, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.91)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 1.05113911413
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 1.0511391141340922, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', None)
Agent drove right instead of left. (rewarded 1.05)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 1.7172189933
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'right'), 'deadline': 6, 't': 19, 'action': None, 'reward': 1.7172189933021722, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.72)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 0.564537225375
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.5645372253751997, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.56)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 1.04188739378
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 4, 't': 21, 'action': None, 'reward': 1.0418873937817184, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.04)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 0.74037121647
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.7403712164700931, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 0.74)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 1.57539306129
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 2, 't': 23, 'action': 'forward', 'reward': 1.5753930612947467, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.58)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 1.67936072465
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 1, 't': 24, 'action': 'right', 'reward': 1.679360724646378, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.68)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 121
\-------------------------
Environment.reset(): Trial set up with start = (3, 2), destination = (8, 3), deadline = 20
Simulating trial. . .
epsilon = 0.2982; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 1.03857246293
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.0385724629285793, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.04)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 2.93591349638
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.9359134963831863, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.94)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 2.8708603473
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.8708603473010244, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.87)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 1.29159343416
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.2915934341559603, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.29)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 2.63714951292
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.6371495129207947, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.64)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 2.31974509879
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 2.3197450987891592, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.32)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.07740540804
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.077405408037233, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.08)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 0.460696818047
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.4606968180467552, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.46)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: forward, reward: 0.218717600639
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 0.2187176006387651, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent drove forward instead of left. (rewarded 0.22)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: forward, reward: -40.2852452183
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': -40.285245218262105, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.29)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 0.918493418356
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': 0.9184934183558511, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 0.92)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 0.608851571908
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.6088515719084546, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.61)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 1.06624109383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.0662410938262663, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.07)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: 1.67223035498
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.6722303549809518, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.67)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 1.38328333747
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.383283337468812, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.38)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 2.28916286176
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.2891628617632627, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.29)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 2.05760253762
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 4, 't': 16, 'action': None, 'reward': 2.0576025376188367, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.06)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: 0.916738979117
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.9167389791174172, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 0.92)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.37086657553
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.3708665755271203, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.37)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.50267137874
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.5026713787448311, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.50)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 122
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (3, 3), deadline = 30
Simulating trial. . .
epsilon = 0.2952; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.64635744963
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.6463574496266762, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.65)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 2.52397341866
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 2.523973418658376, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.52)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: 2.24444076848
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 2.244440768484429, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent followed the waypoint forward. (rewarded 2.24)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: -39.1552623202
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': -39.1552623201693, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.16)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.09474283504
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.0947428350377686, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.09)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: forward, reward: 2.57453770944
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 2.574537709444385, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.57)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: forward, reward: -9.61289977932
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': -9.612899779321248, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.61)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: forward, reward: -9.61147260955
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': -9.61147260955073, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent attempted driving forward through a red light. (rewarded -9.61)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.81942202697
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.819422026970196, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.82)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: left, reward: 0.898510791404
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 0.8985107914042245, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.90)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 2.82409303815
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.8240930381520295, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.82)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 1.32366232318
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.3236623231760023, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.32)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 2.23340531033
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.2334053103283007, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.23)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 0.405863737287
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': None, 'reward': 0.4058637372867018, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.41)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: forward, reward: 2.48010673792
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': 2.4801067379157056, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent followed the waypoint forward. (rewarded 2.48)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 123
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (6, 7), deadline = 30
Simulating trial. . .
epsilon = 0.2923; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2923; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2923; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 1.36700431073
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.3670043107296606, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.37)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: -40.211228386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'right'), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': -40.211228385953234, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'right')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.21)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 0.210515285183
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 0.2105152851826132, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent drove right instead of forward. (rewarded 0.21)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 1.57566042442
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.5756604244195924, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.58)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 2.71777291804
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.717772918037978, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.72)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: left, reward: 2.19913557199
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 2.1991355719896033, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.20)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: 1.71302480353
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.7130248035317694, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.71)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: None, reward: -4.13217302493
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'forward'), 'deadline': 23, 't': 7, 'action': None, 'reward': -4.132173024930524, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.13)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: left, reward: -20.4181329648
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'right'), 'deadline': 22, 't': 8, 'action': 'left', 'reward': -20.418132964782863, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.42)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: left, reward: 0.262710207152
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 0.26271020715223736, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove left instead of forward. (rewarded 0.26)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 2.7602950756
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'forward'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 2.760295075598421, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 2.76)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: 2.36091211845
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'right'), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 2.360912118449975, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'right')
Agent followed the waypoint right. (rewarded 2.36)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: forward, reward: 1.08693277633
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': 1.086932776328544, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.09)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: -0.120877865952
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': -0.12087786595225536, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded -0.12)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: -9.70308250191
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 16, 't': 14, 'action': 'left', 'reward': -9.703082501907051, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent attempted driving left through a red light. (rewarded -9.70)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 2.54055920452
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 15, 'action': None, 'reward': 2.54055920451688, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.54)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: left, reward: 1.71953258368
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 16, 'action': 'left', 'reward': 1.7195325836770583, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.72)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 1.78411702306
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.7841170230577794, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.78)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 0.926274365399
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 18, 'action': None, 'reward': 0.9262743653991838, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.93)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: -0.178260171731
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 11, 't': 19, 'action': 'right', 'reward': -0.17826017173103625, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent drove right instead of left. (rewarded -0.18)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: -9.7209053847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 10, 't': 20, 'action': 'left', 'reward': -9.720905384696405, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.72)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: 0.175243153601
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 0.1752431536009843, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent drove forward instead of right. (rewarded 0.18)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 1.05368537968
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 8, 't': 22, 'action': 'forward', 'reward': 1.0536853796840335, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove forward instead of right. (rewarded 1.05)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 0.948000107495
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 0.9480001074948692, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.95)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: 0.81575226126
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 0.8157522612596226, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 0.82)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 0.954569472798
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 5, 't': 25, 'action': None, 'reward': 0.9545694727976377, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.95)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 1.92568545715
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 26, 'action': None, 'reward': 1.9256854571494624, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.93)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 1.81656153657
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 27, 'action': None, 'reward': 1.8165615365663135, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.82)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 2.07880066663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'right'), 'deadline': 2, 't': 28, 'action': None, 'reward': 2.0788006666284824, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'right')
Agent properly idled at a red light. (rewarded 2.08)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: 1.746817784
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 1, 't': 29, 'action': 'forward', 'reward': 1.7468177840038444, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.75)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 124
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (3, 4), deadline = 25
Simulating trial. . .
epsilon = 0.2894; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2894; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.45778169081
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.457781690809506, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.46)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.78288736578
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.7828873657771203, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.78)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 2.51150874239
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.5115087423903386, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.51)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.84613216806
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.846132168055626, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.85)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.34331924628
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.3433192462758075, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.34)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: left, reward: 2.40989159258
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 2.4098915925772095, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.41)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: 2.57973888938
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 2.5797388893843234, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.58)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.35407542567
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.354075425665876, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.35)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.45751857174
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.4575185717394055, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.46)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: -9.79773491089
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': -9.797734910888005, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.80)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 0.839049675868
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 0.8390496758682942, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.84)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 2.286019799
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.2860197989982742, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.29)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 2.00425924242
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.004259242420542, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.00)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: left, reward: -19.049167791
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': 'left', 'reward': -19.04916779099127, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.05)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: -0.0422556558223
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', 'right'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': -0.04225565582228652, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', 'right')
Agent drove right instead of forward. (rewarded -0.04)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: left, reward: 2.539396721
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 2.539396720996648, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.54)
36% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 125
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (2, 5), deadline = 20
Simulating trial. . .
epsilon = 0.2865; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: right, reward: 2.1189011341
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.1189011341046204, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 2.12)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: right, reward: 2.73432969507
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.7343296950692415, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 2.73)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 2.49921947577
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.4992194757744746, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.50)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 1.86891269667
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.8689126966654421, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.87)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 2.28048212179
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.280482121792729, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.28)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 1.14583827449
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.1458382744892757, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.15)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 2.05160612614
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.0516061261406793, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.05)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 1.58704017873
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.587040178728936, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.59)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 1.41899791619
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.4189979161856268, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.42)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 2.73976922119
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.7397692211912843, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.74)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: -10.5648777319
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': -10.564877731918827, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.56)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 0.286610952934
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.2866109529338299, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.29)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 1.63846325138
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.6384632513819317, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.64)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: 1.45012548242
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.450125482422749, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.45)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 2.1972515489
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.197251548904368, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.20)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 0.959708911437
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.9597089114371715, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.96)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 2.12678614319
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 2.1267861431900115, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.13)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 1.66203745635
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 1.6620374563499363, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.66)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: 0.830182886348
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 0.8301828863483307, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent drove right instead of left. (rewarded 0.83)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: 1.76398020591
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 1.7639802059149006, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.76)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 126
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (3, 7), deadline = 25
Simulating trial. . .
epsilon = 0.2837; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 2.04739195449
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.0473919544904087, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.05)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 2.00410586942
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.0041058694203926, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.00)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 2.48825577823
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 2.488255778232528, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 2.49)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 1.49866174564
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.4986617456418292, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.50)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 2.05489170039
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.0548917003938376, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.05)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 2.45781037832
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 2.457810378324157, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.46)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 2.38040849529
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.380408495288247, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.38)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 0.844440460529
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'forward'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.8444404605294316, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.84)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: left, reward: 2.44822825465
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 2.4482282546469634, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.45)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 1.55440257759
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.5544025775867736, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.55)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 1.20967566584
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.2096756658362267, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.21)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 0.318733829697
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': None, 'reward': 0.31873382969656194, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.32)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: left, reward: 1.37072384018
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 12, 'action': 'left', 'reward': 1.3707238401750466, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.37)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 1.35680740277
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 12, 't': 13, 'action': None, 'reward': 1.3568074027658734, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.36)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: 1.84860974811
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.8486097481109265, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.85)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: -5.06543481296
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'right'), 'deadline': 10, 't': 15, 'action': None, 'reward': -5.065434812956779, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.07)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: right, reward: 1.42836537825
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 1.4283653782514278, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.43)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: 1.18369280477
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', 'forward'), 'deadline': 8, 't': 17, 'action': None, 'reward': 1.1836928047673876, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.18)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: -40.1364388762
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', 'left'), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': -40.13643887624134, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.14)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: right, reward: -0.0655733972353
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': -0.0655733972353193, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent drove right instead of left. (rewarded -0.07)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: forward, reward: -10.100047691
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 5, 't': 20, 'action': 'forward', 'reward': -10.100047690976698, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -10.10)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 0.8905876835
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 4, 't': 21, 'action': None, 'reward': 0.8905876835001512, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.89)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: right, reward: 2.07657075588
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 2.0765707558816047, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.08)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 1.63191391038
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 2, 't': 23, 'action': 'right', 'reward': 1.631913910376323, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.63)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: forward, reward: 1.74737229064
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 1, 't': 24, 'action': 'forward', 'reward': 1.7473722906381888, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.75)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 127
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (5, 5), deadline = 35
Simulating trial. . .
epsilon = 0.2808; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2808; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: forward, reward: 0.903308436165
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 35, 't': 0, 'action': 'forward', 'reward': 0.903308436164737, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent drove forward instead of right. (rewarded 0.90)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 1.07372233391
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 34, 't': 1, 'action': None, 'reward': 1.0737223339148472, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.07)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 2.95895123626
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 33, 't': 2, 'action': None, 'reward': 2.9589512362614925, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.96)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 2.12633959365
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 32, 't': 3, 'action': None, 'reward': 2.1263395936519323, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.13)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 1.29561683541
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 31, 't': 4, 'action': 'right', 'reward': 1.2956168354095512, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.30)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 1.01714648042
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 30, 't': 5, 'action': None, 'reward': 1.0171464804192791, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.02)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 2.315045359
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 29, 't': 6, 'action': None, 'reward': 2.3150453589966618, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.32)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: left, reward: 1.25821699381
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 28, 't': 7, 'action': 'left', 'reward': 1.2582169938071117, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove left instead of forward. (rewarded 1.26)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: left, reward: 1.17703249596
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'left'), 'deadline': 27, 't': 8, 'action': 'left', 'reward': 1.1770324959619263, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'left')
Agent drove left instead of right. (rewarded 1.18)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: forward, reward: 0.996797703019
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 26, 't': 9, 'action': 'forward', 'reward': 0.9967977030189972, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 1.00)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 0.951033605865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', None), 'deadline': 25, 't': 10, 'action': 'right', 'reward': 0.9510336058648549, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', None)
Agent drove right instead of forward. (rewarded 0.95)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.14981757927
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 24, 't': 11, 'action': None, 'reward': 1.149817579267447, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.15)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: left, reward: -10.7724082833
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 23, 't': 12, 'action': 'left', 'reward': -10.772408283271195, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent attempted driving left through a red light. (rewarded -10.77)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: left, reward: 1.97814231797
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 22, 't': 13, 'action': 'left', 'reward': 1.9781423179678914, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 1.98)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 2.42813354922
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 14, 'action': 'forward', 'reward': 2.4281335492228866, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.43)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 0.728942055389
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 20, 't': 15, 'action': 'right', 'reward': 0.7289420553891982, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent drove right instead of forward. (rewarded 0.73)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: left, reward: 2.33872437996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 16, 'action': 'left', 'reward': 2.3387243799647237, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.34)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: left, reward: -20.2567262396
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 18, 't': 17, 'action': 'left', 'reward': -20.256726239628602, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.26)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: left, reward: 2.12039884414
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 17, 't': 18, 'action': 'left', 'reward': 2.1203988441358597, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.12)
46% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 128
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (2, 2), deadline = 20
Simulating trial. . .
epsilon = 0.2780; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2780; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 1.17544474036
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.1754447403590425, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.18)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 2.2959114792
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.295911479202286, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.30)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 2.38219153941
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 2.3821915394125064, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.38)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: -9.37910585267
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': -9.379105852667772, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.38)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 0.930603840805
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.9306038408047231, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.93)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: None, reward: -5.02001820579
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 15, 't': 5, 'action': None, 'reward': -5.020018205791434, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.02)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: right, reward: -0.100102143203
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': -0.10010214320323652, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove right instead of left. (rewarded -0.10)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: left, reward: -20.5629844148
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': -20.562984414820768, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.56)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: right, reward: 1.03808116036
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.0380811603629587, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.04)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: forward, reward: -9.13206427031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': -9.132064270311519, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.13)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: right, reward: 2.52579910801
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 2.5257991080050486, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.53)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: -39.3151318849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': -39.315131884870176, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.32)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 2.64088272711
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.640882727114749, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.64)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 0.319004904144
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.3190049041437323, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.32)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: left, reward: 2.32633024002
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 2.326330240021959, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.33)
25% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 129
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (8, 7), deadline = 35
Simulating trial. . .
epsilon = 0.2753; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: -40.8040335571
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', None), 'deadline': 35, 't': 0, 'action': 'forward', 'reward': -40.80403355712669, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.80)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 2.40582668687
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'right'), 'deadline': 34, 't': 1, 'action': None, 'reward': 2.4058266868735814, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'right')
Agent properly idled at a red light. (rewarded 2.41)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 2.78690584262
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 33, 't': 2, 'action': None, 'reward': 2.786905842621965, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.79)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: right, reward: 1.85449676241
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 32, 't': 3, 'action': 'right', 'reward': 1.8544967624086663, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent drove right instead of left. (rewarded 1.85)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: -4.07094128754
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'right'), 'deadline': 31, 't': 4, 'action': None, 'reward': -4.070941287538994, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.07)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: 1.58465287417
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 30, 't': 5, 'action': 'right', 'reward': 1.5846528741713184, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 1.58)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 2.46031336075
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 29, 't': 6, 'action': None, 'reward': 2.460313360746092, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.46)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 1.97639766991
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 28, 't': 7, 'action': None, 'reward': 1.9763976699068595, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.98)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: right, reward: 1.19319938917
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 27, 't': 8, 'action': 'right', 'reward': 1.1931993891735742, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 1.19)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 1.90019209722
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'right'), 'deadline': 26, 't': 9, 'action': None, 'reward': 1.9001920972235342, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'right')
Agent properly idled at a red light. (rewarded 1.90)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 1.75872682744
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 10, 'action': None, 'reward': 1.7587268274355048, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.76)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 1.87524723394
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 11, 'action': None, 'reward': 1.875247233937138, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.88)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 1.72782572424
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 23, 't': 12, 'action': None, 'reward': 1.727825724243767, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.73)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: forward, reward: 1.07667797751
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 22, 't': 13, 'action': 'forward', 'reward': 1.0766779775087127, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 1.08)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: 1.94963436438
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 21, 't': 14, 'action': 'left', 'reward': 1.9496343643833296, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent followed the waypoint left. (rewarded 1.95)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 2.38411366256
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 20, 't': 15, 'action': 'forward', 'reward': 2.384113662563432, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 2.38)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.1195151221
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 19, 't': 16, 'action': None, 'reward': 2.119515122101756, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.12)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.04834224513
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 18, 't': 17, 'action': None, 'reward': 2.0483422451282025, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.05)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 0.894356585525
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 17, 't': 18, 'action': None, 'reward': 0.894356585525288, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.89)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 0.85089703783
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 16, 't': 19, 'action': 'forward', 'reward': 0.8508970378304841, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.85)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 0.907074346191
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 15, 't': 20, 'action': 'forward', 'reward': 0.90707434619134, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 0.91)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: left, reward: -0.321839089322
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 21, 'action': 'left', 'reward': -0.3218390893219729, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded -0.32)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: left, reward: 2.38045498384
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 13, 't': 22, 'action': 'left', 'reward': 2.3804549838402553, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.38)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: -0.0699254421885
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 23, 'action': 'right', 'reward': -0.0699254421885398, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded -0.07)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.26483383004
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 11, 't': 24, 'action': None, 'reward': 1.2648338300394089, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.26)
29% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: 1.24986616719
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 10, 't': 25, 'action': 'left', 'reward': 1.249866167190841, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove left instead of forward. (rewarded 1.25)
26% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: -19.8657071799
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 9, 't': 26, 'action': 'left', 'reward': -19.865707179869055, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.87)
23% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 1.14484874102
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 27, 'action': 'right', 'reward': 1.1448487410191266, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.14)
20% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 1.12091174073
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 7, 't': 28, 'action': None, 'reward': 1.1209117407286902, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.12)
17% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 1.96466887552
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 6, 't': 29, 'action': 'right', 'reward': 1.9646688755197528, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.96)
14% of time remaining to reach destination.
/-------------------
| Step 30 Results
\-------------------
Environment.step(): t = 30
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: left, reward: 2.10624040447
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 30, 'action': 'left', 'reward': 2.1062404044714915, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.11)
11% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 130
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (4, 2), deadline = 30
Simulating trial. . .
epsilon = 0.2725; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: forward, reward: -40.5547036401
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', 'forward'), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': -40.554703640097735, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.55)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: None, reward: 1.09956154473
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.0995615447334877, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.10)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: None, reward: 1.76916519779
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.769165197788811, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.77)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: None, reward: 2.92994711965
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.929947119648844, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.93)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: None, reward: 0.213043452762
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 0.2130434527624422, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.21)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: right, reward: 1.52324377346
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.523243773463999, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', None)
Agent drove right instead of left. (rewarded 1.52)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: left, reward: 1.02042324515
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 1.0204232451523911, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.02)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: 2.87342070873
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 2.8734207087343933, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.87)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: -4.24257150828
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'right'), 'deadline': 22, 't': 8, 'action': None, 'reward': -4.242571508281582, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.24)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.73278107943
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.7327810794289888, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.73)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.95600866981
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.956008669809476, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.96)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.96420726772
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.9642072677161087, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.96)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.99704731149
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.9970473114899758, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 2.00)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 1.5553766506
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.5553766505978361, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.56)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 1.43315328275
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'right'), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.4331532827453644, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'right')
Agent drove right instead of forward. (rewarded 1.43)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.53321854983
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'left'), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.5332185498346476, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.53)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 2.26689123842
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 14, 't': 16, 'action': None, 'reward': 2.2668912384231934, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.27)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: left, reward: 1.64647598464
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 13, 't': 17, 'action': 'left', 'reward': 1.6464759846378159, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.65)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: -0.0199687323425
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 12, 't': 18, 'action': 'right', 'reward': -0.019968732342537265, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove right instead of forward. (rewarded -0.02)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: None, reward: 2.24681109755
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 11, 't': 19, 'action': None, 'reward': 2.246811097546632, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.25)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: -39.8392267612
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 10, 't': 20, 'action': 'forward', 'reward': -39.83922676123041, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.84)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: left, reward: 2.43366875861
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 9, 't': 21, 'action': 'left', 'reward': 2.4336687586121943, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.43)
27% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 131
\-------------------------
Environment.reset(): Trial set up with start = (2, 2), destination = (1, 5), deadline = 20
Simulating trial. . .
epsilon = 0.2698; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.90420839397
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'right'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.9042083939705015, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'right')
Agent drove forward instead of left. (rewarded 1.90)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 2.23890360562
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.2389036056162945, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.24)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.36282784176
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.3628278417597144, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.36)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: 2.44789001599
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 2.447890015986853, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.45)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: forward, reward: 0.894834311324
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 0.8948343113242624, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.89)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 0.719642081943
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.7196420819434908, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent drove right instead of left. (rewarded 0.72)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 2.00314456403
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.0031445640321346, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.00)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: left, reward: 1.71542985507
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.7154298550695521, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.72)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: right, reward: -0.0629980576412
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'right', 'left'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': -0.0629980576411765, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'right', 'left')
Agent drove right instead of left. (rewarded -0.06)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: 2.05995302798
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.0599530279794562, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.06)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: 2.67335571486
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.6733557148568927, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.67)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: forward, reward: 1.30712031161
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.3071203116106713, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.31)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: None, reward: 2.43926586706
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.439265867058423, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.44)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: forward, reward: 0.732688303839
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.7326883038389462, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 0.73)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 1.98718905346
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 1.9871890534573942, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.99)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.30490469299
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.3049046929910197, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.30)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.66210631481
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.6621063148060928, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.66)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 0.494672618656
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.49467261865555967, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 0.49)
10% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 132
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (7, 7), deadline = 20
Simulating trial. . .
epsilon = 0.2671; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 0.936646055058
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.9366460550580191, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.94)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.71680587375
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.716805873746873, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.72)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.98140460581
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.9814046058067365, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.98)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.96043074791
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.9604307479089396, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.96)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: 2.42742941912
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.4274294191234747, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.43)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 2.10141899927
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.1014189992722505, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 2.10)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 2.36301896674
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.363018966740277, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.36)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: 1.09399725349
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.0939972534907316, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.09)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: 1.38905507642
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'forward'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.3890550764230323, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.39)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: -4.77204116746
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': -4.772041167455685, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.77)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: -0.0873858908145
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': -0.08738589081452075, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent drove forward instead of left. (rewarded -0.09)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: -39.0316683547
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': -39.031668354662855, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.03)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.60080771628
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.6008077162785814, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.60)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: right, reward: 0.956403664048
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.9564036640479173, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove right instead of left. (rewarded 0.96)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: -4.14399388539
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'right'), 'deadline': 6, 't': 14, 'action': None, 'reward': -4.143993885388402, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.14)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: left, reward: -20.8751322571
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'right'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -20.875132257058407, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.88)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 1.91785463224
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.9178546322426429, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.92)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 0.500757486722
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.5007574867222697, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.50)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 2.17616020648
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 2.1761602064773493, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.18)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 1.0510481653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.051048165298573, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.05)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 133
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (4, 2), deadline = 25
Simulating trial. . .
epsilon = 0.2645; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 1.87146743359
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.8714674335878658, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.87)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 2.5770903757
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.5770903757013865, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.58)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 2.67976366086
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.6797636608596154, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.68)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 2.51001097889
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.510010978888845, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.51)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 1.83941271995
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.8394127199489752, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.84)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: left, reward: 1.22121811129
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.2212181112888638, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.22)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: -9.49200400543
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -9.492004005430664, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.49)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 0.0753883952629
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 0.07538839526287155, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.08)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 1.55435452477
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 1.5543545247711885, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.55)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 0.922316017191
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 16, 't': 9, 'action': None, 'reward': 0.922316017190749, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.92)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: -40.074121646
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': -40.07412164598059, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.07)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: -0.182052806113
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': -0.18205280611250496, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded -0.18)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: None, reward: 1.69330383491
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.6933038349146918, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.69)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: 1.01497789012
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 1.0149778901193423, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.01)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.64345025259
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.643450252587324, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.64)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.49801268101
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.4980126810096053, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.50)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: -9.29740735849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 9, 't': 16, 'action': 'left', 'reward': -9.29740735848555, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.30)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 1.64725828583
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.6472582858306717, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.65)
28% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 134
\-------------------------
Environment.reset(): Trial set up with start = (4, 3), destination = (3, 6), deadline = 20
Simulating trial. . .
epsilon = 0.2618; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 2.4561979148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.456197914800902, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.46)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 2.31709407963
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.31709407962581, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.32)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.8351335238
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.8351335238041784, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.84)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 1.71117740917
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.7111774091741192, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.71)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 1.88013077446
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.880130774460983, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.88)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: left, reward: -19.7085395772
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'right'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': -19.70853957719472, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.71)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 1.77862152894
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.7786215289351268, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.78)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 2.57053712596
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 2.5705371259625296, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.57)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: forward, reward: 1.94584169633
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'right'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.9458416963272398, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'right')
Agent followed the waypoint forward. (rewarded 1.95)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 1.51642558525
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.5164255852518167, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 1.52)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: -5.08111232518
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': -5.081112325179219, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.08)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: left, reward: 2.15004351541
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 2.1500435154055357, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.15)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: 1.07271610967
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 1.0727161096715014, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.07)
35% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 135
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (2, 5), deadline = 20
Simulating trial. . .
epsilon = 0.2592; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2592; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2592; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.78204862912
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.7820486291153859, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.78)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 2.79710937883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.7971093788336896, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.80)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 2.20518697208
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'right'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.20518697207833, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'right')
Agent properly idled at a red light. (rewarded 2.21)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.99789669887
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.9978966988680786, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.00)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: -9.05593740763
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': -9.055937407633643, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.06)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 0.923492613186
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 0.9234926131862695, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.92)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 0.910917452106
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 0.9109174521063033, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.91)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 2.7800663697
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.78006636970261, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.78)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.51393789725
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.5139378972469462, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.51)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: left, reward: 1.23893058608
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.238930586083343, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.24)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: right, reward: 1.68841972773
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.6884197277271586, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.69)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: forward, reward: 1.45648139286
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'right'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.4564813928575284, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'right')
Agent drove forward instead of right. (rewarded 1.46)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: left, reward: -39.61533595
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -39.615335950013375, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.62)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 2.55205277781
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 2.552052777812503, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.55)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 136
\-------------------------
Environment.reset(): Trial set up with start = (7, 6), destination = (6, 3), deadline = 20
Simulating trial. . .
epsilon = 0.2567; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2567; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2567; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2567; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2567; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2567; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2567; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 1.58593206572
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.5859320657202953, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.59)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 1.09604913262
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.0960491326234891, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.10)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 1.03981156859
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.0398115685870082, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.04)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: left, reward: 2.68419748647
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 2.6841974864734968, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.68)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 2.27838473326
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.278384733263139, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.28)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 1.7296123756
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.7296123756042299, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.73)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: 1.98609634499
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.986096344988703, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.99)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: left, reward: 2.30292826933
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 2.3029282693331634, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.30)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 1.67559679101
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.6755967910149243, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.68)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 137
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (4, 3), deadline = 20
Simulating trial. . .
epsilon = 0.2541; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: forward, reward: 1.67340804097
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.673408040966528, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.67)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 1.41551827244
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.4155182724422364, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove right instead of left. (rewarded 1.42)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: left, reward: 2.36931809887
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 2.369318098870565, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.37)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: left, reward: 2.63722219284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 2.6372221928441424, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.64)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 0.872606358147
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.8726063581474138, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove right instead of forward. (rewarded 0.87)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: left, reward: 1.48534853275
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.4853485327492972, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.49)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: 2.54650713298
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 2.54650713297898, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.55)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: right, reward: 2.04921778333
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 2.0492177833275607, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.05)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 138
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (2, 6), deadline = 30
Simulating trial. . .
epsilon = 0.2516; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: forward, reward: 1.33705699469
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'right'), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': 1.3370569946943145, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'right')
Agent followed the waypoint forward. (rewarded 1.34)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: 1.03730194712
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 1.0373019471224567, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.04)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: right, reward: 0.243933468166
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'left'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 0.2439334681659332, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'left')
Agent drove right instead of forward. (rewarded 0.24)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: left, reward: 2.27305073184
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'left', 'reward': 2.2730507318439734, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.27)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: 1.78065445608
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 1.7806544560802142, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.78)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: -39.3196604256
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': -39.31966042555751, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.32)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 2.91665008538
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.916650085375463, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.92)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: -9.64704592435
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': -9.647045924347765, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.65)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: -9.36045962847
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': -9.36045962846602, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.36)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 0.94290376045
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 0.9429037604496262, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.94)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.73956534841
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.7395653484094549, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.74)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 2.78457172267
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.784571722665002, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.78)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 2.62328117949
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.623281179494982, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.62)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 1.34724341388
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 1.3472434138783425, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.35)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: -9.12612576294
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': -9.126125762937122, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.13)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.94888287652
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.9488828765165855, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.95)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 2.68039853458
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'left'), 'deadline': 14, 't': 16, 'action': None, 'reward': 2.680398534582734, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.68)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: forward, reward: 2.68696929241
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 13, 't': 17, 'action': 'forward', 'reward': 2.6869692924115123, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.69)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 139
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (3, 7), deadline = 20
Simulating trial. . .
epsilon = 0.2491; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.33938488625
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.3393848862502757, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.34)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.27150893301
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.271508933005981, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.27)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.30290510425
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.3029051042476676, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.30)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 1.38508306954
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.3850830695365557, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 1.39)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: 0.00365190129477
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 0.0036519012947674456, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 0.00)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 0.925480210509
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.9254802105092657, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.93)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 1.91383525387
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.9138352538665273, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.91)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 2.02996083507
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.029960835065049, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.03)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 1.2447539781
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.2447539780982741, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.24)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 1.18741744978
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.187417449777492, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.19)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 2.04007665407
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 2.04007665406934, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.04)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 140
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (6, 2), deadline = 20
Simulating trial. . .
epsilon = 0.2466; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 1.15955233869
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.159552338685252, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.16)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 2.15608693516
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.1560869351579512, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 2.16)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 1.6199877997
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.6199877996978285, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.62)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 2.4333697873
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.4333697873023183, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.43)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: forward, reward: 0.189420333716
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 0.1894203337158057, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.19)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: 1.34389328711
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.3438932871086402, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded 1.34)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: 0.281541627971
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 0.28154162797065796, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 0.28)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 1.42487632365
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.4248763236494901, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.42)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.04692184933
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.0469218493275068, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.05)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 0.916974248197
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.9169742481965113, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 0.92)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 0.167380721831
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.16738072183107489, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.17)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: -9.24069941233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': -9.240699412329565, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.24)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 2.40691123424
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.4069112342404724, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.41)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 1.97417275169
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.97417275169342, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.97)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 2.42521462579
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.4252146257864897, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.43)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 1.56926593733
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.5692659373262308, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.57)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: left, reward: 1.50329730781
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 1.5032973078099692, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.50)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 2.0425712579
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 2.0425712578958364, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.04)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: -9.51134189883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': -9.511341898825288, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.51)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 1.49958095478
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'right'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 1.4995809547813868, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, 'right', 'right')
Agent followed the waypoint forward. (rewarded 1.50)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 141
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (7, 5), deadline = 30
Simulating trial. . .
epsilon = 0.2441; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: -4.53947339584
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 30, 't': 0, 'action': None, 'reward': -4.539473395843384, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.54)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: -5.24469468869
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': -5.244694688693856, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.24)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: 1.85704571636
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'right'), 'deadline': 28, 't': 2, 'action': 'right', 'reward': 1.8570457163599055, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'right')
Agent followed the waypoint right. (rewarded 1.86)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 1.41206019734
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 1.4120601973362301, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.41)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: left, reward: 1.09400732096
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 1.0940073209583536, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.09)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 1.77072732988
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.7707273298831288, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.77)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.1107934793
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.1107934793003844, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.11)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: left, reward: 2.23842152904
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 2.238421529037988, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.24)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 2.1863788988
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 8, 'action': None, 'reward': 2.186378898795555, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.19)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 1.20547209166
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 1.2054720916599218, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.21)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 1.65246066977
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 20, 't': 10, 'action': None, 'reward': 1.6524606697726398, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 1.65)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 2.22278342952
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.2227834295183175, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.22)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 1.6096980099
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.6096980098987987, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.61)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: left, reward: 1.22427460277
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 1.2242746027694624, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.22)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 1.58067268395
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.5806726839476046, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.58)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 1.40243134571
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 1.4024313457144832, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.40)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 2.08484741351
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 14, 't': 16, 'action': None, 'reward': 2.0848474135129638, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.08)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 2.54623119114
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 2.5462311911426467, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.55)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.05586421681
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.0558642168128938, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.06)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: -5.38280778722
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 19, 'action': None, 'reward': -5.382807787222214, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.38)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 0.617401115517
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 10, 't': 20, 'action': 'forward', 'reward': 0.617401115517275, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.62)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 2.23271743343
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 2.2327174334336153, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.23)
27% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 142
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (3, 2), deadline = 25
Simulating trial. . .
epsilon = 0.2417; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2417; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.11223222809
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.1122322280919719, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.11)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.19380322655
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.193803226550869, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.19)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: -9.52679394819
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': -9.526793948190432, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.53)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 1.17906803313
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.1790680331336207, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.18)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.62762867496
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.627628674956636, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.63)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 2.61645681595
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.616456815953915, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.62)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: -0.0236012502462
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': -0.0236012502462416, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded -0.02)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 2.79039727788
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'left', 'forward'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 2.7903972778785473, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.79)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.00208422291
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.002084222911207, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.00)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 1.12449580765
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.1244958076471394, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.12)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 1.51219467615
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 1.5121946761483185, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.51)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.34618105048
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.346181050483701, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.35)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.28456668502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.284566685016948, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.28)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 2.63829385819
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 2.638293858192201, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.64)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: left, reward: -0.286413937099
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -0.2864139370989306, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded -0.29)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 1.05115273891
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.0511527389086335, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.05)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: -9.87298530548
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': -9.872985305480043, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.87)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 1.00017115545
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.0001711554500523, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.00)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: forward, reward: -0.0684157881601
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': -0.06841578816012417, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded -0.07)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 0.544836461267
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 6, 't': 19, 'action': None, 'reward': 0.5448364612666656, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 0.54)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: left, reward: -9.15737972713
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 5, 't': 20, 'action': 'left', 'reward': -9.157379727125077, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -9.16)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: right, reward: 2.24888303687
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 2.2488830368681665, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.25)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 0.7975747882
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 0.7975747881999373, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 0.80)
8% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 143
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (2, 7), deadline = 30
Simulating trial. . .
epsilon = 0.2393; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2393; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: 1.53498345226
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 1.5349834522579877, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.53)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 2.51678392756
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.516783927559705, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.52)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 1.16934162391
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.1693416239113932, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.17)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 1.55743827567
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.5574382756698977, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.56)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: left, reward: 0.947176881364
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 0.9471768813635476, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove left instead of forward. (rewarded 0.95)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 2.76325543321
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 2.7632554332093946, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.76)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 0.970576429105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 24, 't': 6, 'action': None, 'reward': 0.9705764291045098, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.97)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.25759225105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.2575922510483644, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.26)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: -9.82983835523
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': -9.829838355231406, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.83)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: 0.415194410513
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 0.4151944105127341, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent drove left instead of forward. (rewarded 0.42)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 0.916270887094
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 0.9162708870944476, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.92)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 1.81339752559
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.813397525585613, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.81)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 1.35276378524
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.352763785241195, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.35)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 2.70402520169
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 2.704025201692712, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.70)
53% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 144
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (4, 7), deadline = 20
Simulating trial. . .
epsilon = 0.2369; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: None, reward: -4.62626947133
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': -4.6262694713312715, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.63)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 1.22082910662
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.220829106623924, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded 1.22)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.8664414161
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.8664414160991103, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.87)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.39488372846
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.3948837284559237, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.39)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 0.981622779428
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.9816227794278958, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 0.98)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: -19.857636892
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': -19.85763689195113, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.86)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 2.71341894259
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.7134189425941226, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 2.71)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.04741388917
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'forward'), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.0474138891728957, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.05)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.17220646261
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.1722064626090138, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.17)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.01966082438
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.0196608243754963, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.02)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.6941292241
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.6941292241018782, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.69)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.18909403927
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'right'), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.1890940392726395, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'right')
Agent properly idled at a red light. (rewarded 1.19)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.26196152092
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.2619615209249049, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.26)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: 1.28416661179
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.2841666117924002, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove left instead of forward. (rewarded 1.28)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 1.3220689135
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.3220689135023493, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.32)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 1.23896561473
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.2389656147270038, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.24)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 145
\-------------------------
Environment.reset(): Trial set up with start = (5, 7), destination = (2, 2), deadline = 20
Simulating trial. . .
epsilon = 0.2346; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: left, reward: -39.6383943253
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -39.638394325334474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.64)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: -10.5523815795
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -10.552381579516835, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent attempted driving forward through a red light. (rewarded -10.55)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.23539816031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.235398160310269, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.24)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 1.53384925965
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.5338492596498032, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.53)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 1.82782376559
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.8278237655939644, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.83)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 0.717120733909
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'right'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.717120733908778, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'right')
Agent drove right instead of forward. (rewarded 0.72)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: -5.36461793689
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 14, 't': 6, 'action': None, 'reward': -5.364617936893511, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.36)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 2.05525662355
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.055256623554621, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 2.06)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: -9.57030650904
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'forward'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -9.570306509037904, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.57)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: 2.07522777121
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 2.075227771206771, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.08)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: forward, reward: 2.36445797909
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 2.364457979093363, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.36)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: right, reward: 0.365605508265
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.3656055082650531, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 0.37)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 0.835685821762
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.8356858217617305, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent followed the waypoint right. (rewarded 0.84)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: left, reward: 1.2200060145
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'right', 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.2200060145046083, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'right', 'left')
Agent drove left instead of right. (rewarded 1.22)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 2.03247400096
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.032474000962231, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.03)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: right, reward: -0.445458717454
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': -0.4454587174544966, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent drove right instead of left. (rewarded -0.45)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: left, reward: 2.39872618947
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 2.398726189468304, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.40)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: left, reward: 1.14491300587
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 1.1449130058683583, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.14)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: 1.71638346163
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 1.7163834616260276, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.72)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: -0.156968677458
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -0.15696867745846188, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', 'right', None, None)
Agent drove forward instead of right. (rewarded -0.16)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 146
\-------------------------
Environment.reset(): Trial set up with start = (2, 6), destination = (5, 3), deadline = 30
Simulating trial. . .
epsilon = 0.2322; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2322; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2322; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: 1.9735736494
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 1.9735736494026181, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.97)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: forward, reward: 1.09587077164
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 1.0958707716384202, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent followed the waypoint forward. (rewarded 1.10)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: forward, reward: 2.48203013849
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 2.4820301384901335, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 2.48)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: left, reward: 1.64395729653
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 27, 't': 3, 'action': 'left', 'reward': 1.6439572965288596, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove left instead of right. (rewarded 1.64)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: left, reward: 1.50850388037
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 1.5085038803669155, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 1.51)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 1.88726317862
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'right'), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.8872631786219765, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 1.89)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 1.7491513982
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.7491513982011295, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.75)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: 1.18644527155
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'left'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.1864452715490448, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 1.19)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: forward, reward: -10.2591693187
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': -10.25916931873184, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.26)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 1.76070585855
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.760705858548026, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.76)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: left, reward: 2.288139252
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 2.2881392519962978, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.29)
63% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 147
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (4, 4), deadline = 20
Simulating trial. . .
epsilon = 0.2299; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2299; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2299; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 0.0368020422012
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 0.03680204220116723, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.04)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 0.722362792398
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.7223627923977173, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 0.72)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: forward, reward: 1.95209624232
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.9520962423187056, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 1.95)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 2.24397946357
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.243979463569671, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.24)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 1.42069867044
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.4206986704423195, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.42)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 1.02896614018
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.0289661401848926, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.03)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 2.33701303702
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.3370130370213693, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.34)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: 2.65610108015
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.6561010801485923, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.66)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 2.65967483154
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.659674831542749, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.66)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 1.4161991407
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.4161991406975794, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.42)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: right, reward: 1.30356272896
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.303562728961427, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.30)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: right, reward: 2.03532563981
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 2.0353256398134687, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.04)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: right, reward: 0.680716925729
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.680716925729336, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 0.68)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 2.10843539923
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 2.108435399226601, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.11)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 2.27866469634
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.2786646963400137, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.28)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: left, reward: -10.5661777702
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -10.566177770218065, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.57)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: forward, reward: 1.26361910783
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.263619107834121, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.26)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 0.73259461777
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.732594617769518, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 0.73)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 0.900510378916
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.9005103789158404, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.90)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: right, reward: -0.321155508721
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'right'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': -0.3211555087207839, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'forward', 'left', 'right')
Agent drove right instead of left. (rewarded -0.32)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 148
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (4, 7), deadline = 25
Simulating trial. . .
epsilon = 0.2276; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2276; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2276; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: None, reward: 2.08292211519
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.0829221151945916, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.08)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: None, reward: 1.11032179119
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.1103217911861145, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.11)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: None, reward: 2.27134641539
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.2713464153881797, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.27)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: left, reward: 2.30391378338
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 2.3039137833846035, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.30)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 2.76446730705
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.7644673070471377, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.76)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 2.18552284493
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.1855228449337245, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.19)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 2.35782633171
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.357826331707309, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.36)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: 1.29042139852
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.2904213985226145, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.29)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 1.44686356538
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.446863565383393, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.45)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: forward, reward: 1.08853441066
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.0885344106609502, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent followed the waypoint forward. (rewarded 1.09)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 0.913552312181
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 15, 't': 10, 'action': None, 'reward': 0.9135523121805327, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 0.91)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 1.37721571425
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.3772157142467252, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.38)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: 1.20615903824
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.2061590382351906, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.21)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: forward, reward: 1.33024390146
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 1.330243901455992, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 1.33)
44% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 149
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (2, 2), deadline = 25
Simulating trial. . .
epsilon = 0.2254; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.39483640902
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.3948364090240903, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.39)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.52807856566
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.528078565657466, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.53)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: -9.32854888333
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': 'left', 'reward': -9.328548883326372, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.33)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.42686361663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.4268636166322013, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.43)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: 0.957292854176
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 0.9572928541757257, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.96)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 1.85854947108
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.8585494710786865, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.86)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 2.13353757677
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.133537576767681, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.13)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 0.603488513281
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.6034885132806389, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent drove right instead of left. (rewarded 0.60)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 1.06581134766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.0658113476603166, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent drove right instead of forward. (rewarded 1.07)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 2.04791967343
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.047919673429481, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.05)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 1.56798787822
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.5679878782178516, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.57)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: 1.94450851738
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.9445085173767673, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.94)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 1.72199589376
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.7219958937555853, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove right instead of forward. (rewarded 1.72)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 2.17183045785
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 2.1718304578499086, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.17)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 1.51275256299
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.5127525629929157, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.51)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: 2.3550039455
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 2.3550039454964944, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.36)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: left, reward: 0.688894239231
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 0.6888942392313476, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove left instead of forward. (rewarded 0.69)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 1.03176507026
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 1.0317650702564347, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.03)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: left, reward: 1.44165590038
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 18, 'action': 'left', 'reward': 1.4416559003840088, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.44)
24% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 150
\-------------------------
Environment.reset(): Trial set up with start = (7, 2), destination = (8, 5), deadline = 20
Simulating trial. . .
epsilon = 0.2231; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2231; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2231; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 2.13378621168
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.133786211684979, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.13)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.66031512846
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.66031512845959, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.66)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.19132195045
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.1913219504515986, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.19)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: -10.9054489807
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': -10.905448980672052, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.91)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 2.85442095999
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.8544209599882193, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.85)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 2.27986948972
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.2798694897236658, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.28)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: right, reward: 1.80896753999
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.8089675399945802, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent drove right instead of left. (rewarded 1.81)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 0.984110456997
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.9841104569973733, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 0.98)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 2.53830416486
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.5383041648605, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 2.54)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 0.915961745116
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.9159617451159545, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.92)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: left, reward: 2.5266995164
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 2.526699516398661, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.53)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.51985889715
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.519858897148727, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.52)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 0.671703878397
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.6717038783970393, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 0.67)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: 0.696588458537
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.6965884585374618, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 0.70)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: -0.257728957562
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': -0.25772895756158254, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent drove left instead of forward. (rewarded -0.26)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 1.04910658508
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.0491065850835215, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.05)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 0.731553030549
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.7315530305486162, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 0.73)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 1.496912736
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.49691273600089, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.50)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 1.86607505579
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.8660750557908548, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.87)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 0.380397624356
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 0.3803976243559505, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.38)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 151
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (2, 3), deadline = 20
Simulating trial. . .
epsilon = 0.2209; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2209; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 2.48055653501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.4805565350077035, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.48)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 2.25047463448
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.2504746344771216, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.25)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 1.79384406973
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.7938440697340723, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.79)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 1.75532891432
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.7553289143205357, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.76)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 2.36778054677
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.3677805467679858, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.37)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: forward, reward: 0.986682033963
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 0.9866820339634843, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.99)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: left, reward: -9.06729662453
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': -9.067296624526573, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.07)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 1.1902915915
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.1902915914966232, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.19)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 2.4874947073
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.4874947072973876, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.49)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 2.28901161658
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.2890116165796064, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.29)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 1.41899539736
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.4189953973614031, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent drove right instead of left. (rewarded 1.42)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 1.96404690491
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.9640469049082168, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 1.96)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: left, reward: 0.918062850803
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 0.9180628508032327, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove left instead of forward. (rewarded 0.92)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 1.80981681314
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.8098168131402113, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.81)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.43720149471
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.437201494709237, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.44)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: left, reward: 0.815381623898
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 0.8153816238979306, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.82)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 0.910729294992
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.9107292949918699, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent drove right instead of forward. (rewarded 0.91)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: -5.28335763626
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 3, 't': 17, 'action': None, 'reward': -5.283357636256518, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.28)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: -4.99680488033
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': -4.996804880334663, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.00)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 0.164528021207
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.1645280212069019, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent drove right instead of left. (rewarded 0.16)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 152
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (8, 4), deadline = 20
Simulating trial. . .
epsilon = 0.2187; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.36190849957
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.361908499574477, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.36)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.12971247294
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.1297124729433259, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.13)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.02619574102
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.0261957410190745, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.03)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 0.980033736867
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 0.9800337368665417, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.98)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.24853794339
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.2485379433943173, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.25)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: left, reward: 1.99943996612
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.9994399661185116, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 2.00)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.20060226098
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.2006022609812392, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.20)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: 1.42321234501
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.423212345005622, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.42)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 2.27082466556
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.2708246655629383, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.27)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 2.13264298251
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.132642982508618, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.13)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: 1.19078474643
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.1907847464272756, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.19)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 1.83367341644
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.8336734164396191, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.83)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: left, reward: -40.4398751509
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': -40.43987515094043, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.44)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: right, reward: 1.59912038595
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.5991203859485907, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.60)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 153
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (7, 2), deadline = 20
Simulating trial. . .
epsilon = 0.2165; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: left, reward: 2.30846923332
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.30846923331801, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.31)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 1.67726923501
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'right'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.6772692350095975, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'right')
Agent drove right instead of forward. (rewarded 1.68)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 0.0361033871209
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.03610338712093386, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent drove right instead of left. (rewarded 0.04)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.57835892391
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.5783589239063371, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.58)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 0.111702227457
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 0.11170222745682279, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.11)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 2.17107839937
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.1710783993707254, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.17)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.92199143911
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.9219914391132775, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.92)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 2.32937080293
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.329370802926608, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 2.33)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 0.912894143857
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 0.912894143857117, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.91)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 0.927830207118
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.9278302071177087, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove right instead of left. (rewarded 0.93)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 1.67603861549
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.6760386154909392, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.68)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.84146491603
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.8414649160328311, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.84)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.68650243506
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.686502435063068, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.69)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.68405438916
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.6840543891615303, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.68)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.5536286657
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.553628665700166, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.55)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: -0.260055407378
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': -0.26005540737804234, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded -0.26)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 2.33386562083
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 4, 't': 16, 'action': None, 'reward': 2.3338656208269857, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.33)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 1.66784646853
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.6678464685338756, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.67)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 0.326229018611
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.326229018610815, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.33)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: left, reward: 0.516258488908
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 0.5162584889078814, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.52)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 154
\-------------------------
Environment.reset(): Trial set up with start = (7, 2), destination = (1, 4), deadline = 20
Simulating trial. . .
epsilon = 0.2144; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2144; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 2.39613882526
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.3961388252556857, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.40)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: 1.05878107332
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.0587810733160712, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.06)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 1.77411397651
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.774113976507885, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove right instead of left. (rewarded 1.77)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 1.54513447824
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.5451344782423242, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.55)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: forward, reward: 1.85641371367
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.856413713665253, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 1.86)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: -0.0212043821471
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': -0.021204382147122613, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.02)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: left, reward: 2.21895542213
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.2189554221338126, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.22)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 1.33186122757
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.3318612275728805, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.33)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: right, reward: 1.37290308508
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.372903085083947, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent drove right instead of left. (rewarded 1.37)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: 2.42788930833
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 2.427889308331504, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.43)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 0.829148314641
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 0.8291483146407561, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 0.83)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 2.65055215979
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.6505521597927437, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.65)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 1.58493726807
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'right'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.584937268074359, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'right')
Agent properly idled at a red light. (rewarded 1.58)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 2.26901845772
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 2.269018457715366, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 2.27)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 0.784675006099
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 0.7846750060986258, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 0.78)
25% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 155
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (4, 6), deadline = 20
Simulating trial. . .
epsilon = 0.2122; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2122; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2122; alpha = 0.5000
Simulating trial. . .
epsilon = 0.2122; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: -4.92182719846
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': -4.921827198461584, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -4.92)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: 1.60057691468
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.600576914681147, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 1.60)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 2.9634910069
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.9634910069036433, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.96)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 1.00311368825
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.0031136882460587, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 1.00)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: 0.298723295411
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.2987232954107949, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.30)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 2.90931837818
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.909318378179096, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.91)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: left, reward: -10.748392937
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': -10.74839293702222, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.75)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: -4.34474173989
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': -4.344741739892873, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.34)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: left, reward: 2.20910418293
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 2.209104182926499, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.21)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 1.76801452814
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.7680145281372925, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 1.77)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: left, reward: 1.17068817241
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.1706881724144673, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.17)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 156
\-------------------------
Environment.reset(): Trial set up with start = (5, 5), destination = (8, 4), deadline = 20
Simulating trial. . .
epsilon = 0.2101; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: forward, reward: -40.5936638254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -40.59366382540525, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.59)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: right, reward: 1.7423378464
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.742337846398403, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.74)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: None, reward: 2.85410894506
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.8541089450560033, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.85)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: forward, reward: 1.11078605102
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.1107860510188245, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.11)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 2.35078652453
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.350786524534045, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.35)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 1.86851942486
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.8685194248569743, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.87)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.30463822696
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.3046382269646475, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.30)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 0.684286978562
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.6842869785621428, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove right instead of left. (rewarded 0.68)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 2.4702840269
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.4702840269041917, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.47)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.85255263705
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.8525526370483012, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.85)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: -0.118660184652
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': -0.11866018465240769, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent drove forward instead of right. (rewarded -0.12)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 1.19354364297
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.1935436429670125, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.19)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 0.980725622766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'right'), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.9807256227659409, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 0.98)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 1.0761332848
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.0761332847994902, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.08)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 0.607210984246
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 0.6072109842456201, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.61)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: left, reward: 1.68748373161
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 1.6874837316075273, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.69)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 157
\-------------------------
Environment.reset(): Trial set up with start = (5, 2), destination = (1, 6), deadline = 30
Simulating trial. . .
epsilon = 0.2080; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 1.42105137334
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.4210513733443815, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.42)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: left, reward: -9.87210365492
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': 'left', 'reward': -9.87210365491893, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.87)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 2.74374255972
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.7437425597177905, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.74)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 1.29238824018
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 1.2923882401793327, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.29)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 2.17620184711
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.176201847105494, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.18)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: right, reward: 2.78625291451
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 2.7862529145119237, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.79)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: left, reward: 1.79949662712
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', None), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 1.7994966271217332, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', None)
Agent drove left instead of forward. (rewarded 1.80)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 1.68961647662
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.6896164766180466, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.69)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: 2.52959060043
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': 2.529590600432228, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 2.53)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: 1.13183544394
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 1.1318354439417393, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent drove right instead of forward. (rewarded 1.13)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 2.12626131171
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.1262613117100333, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.13)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: right, reward: 1.29558898086
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 1.295588980857552, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.30)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 1.83911769737
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 1.8391176973700523, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.84)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 2.46467193681
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 13, 'action': None, 'reward': 2.464671936805699, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.46)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 1.19205398384
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.1920539838448856, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.19)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 1.80688818078
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.8068881807817354, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.81)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 1.40680405364
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': 1.406804053644049, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.41)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: -39.8447697791
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 13, 't': 17, 'action': 'left', 'reward': -39.844769779057216, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.84)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 1.07586348981
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 12, 't': 18, 'action': None, 'reward': 1.075863489813536, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.08)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: 0.62248911557
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 0.62248911557047, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent drove right instead of forward. (rewarded 0.62)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.61506034141
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 10, 't': 20, 'action': None, 'reward': 1.6150603414077893, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.62)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: 1.9364454567
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 9, 't': 21, 'action': 'left', 'reward': 1.9364454567048779, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.94)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 1.52925601305
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 8, 't': 22, 'action': None, 'reward': 1.5292560130456303, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.53)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: -0.345290050846
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 7, 't': 23, 'action': 'forward', 'reward': -0.34529005084645725, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded -0.35)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: left, reward: 1.89929883914
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 6, 't': 24, 'action': 'left', 'reward': 1.8992988391379884, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.90)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: 1.13715625001
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 1.1371562500138952, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 1.14)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.01633303441
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 4, 't': 26, 'action': 'right', 'reward': 1.0163330344106074, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.02)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: forward, reward: -9.949401169
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 3, 't': 27, 'action': 'forward', 'reward': -9.949401169001929, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.95)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 0.916192248924
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 2, 't': 28, 'action': 'right', 'reward': 0.9161922489241796, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.92)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 1.36321173478
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 1, 't': 29, 'action': 'forward', 'reward': 1.3632117347765569, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.36)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 158
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (8, 5), deadline = 25
Simulating trial. . .
epsilon = 0.2060; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: right, reward: 2.69044790132
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.690447901316836, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.69)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.76455041183
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.764550411827128, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.76)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: -9.2504754069
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': -9.250475406896495, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -9.25)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 1.73713581835
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.737135818345439, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.74)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: None, reward: 2.83621745071
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.836217450712507, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.84)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: forward, reward: 1.72794292851
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.7279429285144439, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.73)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: 1.83099574973
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.8309957497260378, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.83)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.14535127611
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.1453512761107916, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.15)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: -10.0804882584
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -10.080488258392505, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent attempted driving forward through a red light. (rewarded -10.08)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: 1.75548891133
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.7554889113339278, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.76)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 0.942387482172
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.9423874821716967, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 0.94)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 159
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (4, 7), deadline = 25
Simulating trial. . .
epsilon = 0.2039; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 2.37211640815
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.3721164081506805, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.37)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 1.79051379436
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.7905137943624079, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.79)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 2.52641705257
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.526417052568937, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.53)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 2.32594968624
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.3259496862355675, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.33)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 2.66122837118
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 2.661228371178597, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.66)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.73461844704
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.73461844704107, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.73)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.4850316376
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.485031637596875, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.49)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 0.234278021363
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.2342780213631801, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent drove right instead of forward. (rewarded 0.23)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: left, reward: 2.12716101633
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': 2.1271610163279417, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.13)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: 1.09026066298
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.090260662980661, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 1.09)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: -39.1724795668
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': -39.172479566768885, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.17)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.69109906408
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.6910990640825911, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.69)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 2.51492853898
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.514928538975113, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.51)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: left, reward: 0.146514398756
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 0.14651439875560324, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.15)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: left, reward: -40.239319018
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'right'), 'deadline': 11, 't': 14, 'action': 'left', 'reward': -40.23931901801946, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.24)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 1.721755144
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.721755143998672, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.72)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 1.97393390046
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 1.9739339004609595, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.97)
32% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 160
\-------------------------
Environment.reset(): Trial set up with start = (2, 6), destination = (4, 2), deadline = 20
Simulating trial. . .
epsilon = 0.2019; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 1.55853872988
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.558538729878799, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.56)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 0.393263172108
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.39326317210805806, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.39)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.59109238909
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.5910923890933382, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.59)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 2.70794509863
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.707945098625859, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.71)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: right, reward: 0.471915002967
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.4719150029669673, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent drove right instead of left. (rewarded 0.47)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: left, reward: 1.83674487541
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.8367448754086657, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.84)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 0.156053337028
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.15605333702770907, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', None)
Agent drove right instead of left. (rewarded 0.16)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 2.81796792189
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 2.81796792188913, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.82)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.93999464789
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.9399946478946102, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.94)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.12361428754
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.123614287542897, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.12)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.92072259663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.9207225966330448, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.92)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 2.35536884671
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 2.355368846712394, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.36)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 2.2237440709
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.2237440708977707, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.22)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 0.682095738383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.682095738383031, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.68)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: forward, reward: 0.498457754305
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 0.49845775430487016, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent drove forward instead of left. (rewarded 0.50)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 0.841744159932
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.8417441599323554, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', 'left')
Agent properly idled at a red light. (rewarded 0.84)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 0.339882823969
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'left'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.33988282396901814, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'left')
Agent drove right instead of left. (rewarded 0.34)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 0.679591254499
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.6795912544992229, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 0.68)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 0.892915785153
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.8929157851532648, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.89)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 1.04028640552
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 1.04028640551632, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.04)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 161
\-------------------------
Environment.reset(): Trial set up with start = (3, 2), destination = (7, 6), deadline = 30
Simulating trial. . .
epsilon = 0.1999; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: forward, reward: 0.00373014737948
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'forward'), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': 0.0037301473794792983, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.00)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: forward, reward: 0.654752643644
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'right'), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 0.6547526436443266, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'right')
Agent drove forward instead of left. (rewarded 0.65)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: left, reward: 2.39377880805
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 28, 't': 2, 'action': 'left', 'reward': 2.3937788080504627, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.39)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 1.74594721245
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.745947212450743, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.75)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 2.14474362933
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.144743629328148, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.14)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: 1.49756578603
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.4975657860316998, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.50)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.84931716504
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.8493171650387634, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.85)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.58861038055
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.5886103805454852, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.59)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.69223436471
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.6922343647058662, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.69)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: left, reward: 0.526122633636
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 0.5261226336363283, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 0.53)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 1.59010583235
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.5901058323530526, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.59)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 1.9642051407
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.9642051406952692, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.96)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 1.61573748273
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 1.615737482729025, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.62)
57% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 162
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (2, 3), deadline = 25
Simulating trial. . .
epsilon = 0.1979; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 2.29830654045
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.298306540451435, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.30)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: forward, reward: -10.9164051955
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, 'forward'), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': -10.916405195486162, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.92)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 2.25862702831
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.2586270283061185, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.26)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 1.96977484322
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.9697748432240418, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.97)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 2.32147638819
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.321476388185664, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.32)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: 2.57343352194
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 2.573433521940705, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.57)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: left, reward: 1.77120830593
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.7712083059342776, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent drove left instead of forward. (rewarded 1.77)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 2.36638551234
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 2.3663855123379123, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.37)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.54269910178
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.5426991017782277, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.54)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.53958622784
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.5395862278390327, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.54)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 1.14846247759
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.148462477588871, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.15)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: 1.9022187981
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.9022187981010048, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.90)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 2.36937519165
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 2.3693751916529413, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.37)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 2.66581694843
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 2.665816948425187, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.67)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 0.986680925835
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 11, 't': 14, 'action': None, 'reward': 0.9866809258349811, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.99)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: forward, reward: -9.54642261526
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': -9.54642261526399, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.55)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: forward, reward: -9.22076713097
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': -9.220767130965749, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.22)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: 1.31943636884
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 1.319436368842171, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.32)
28% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 163
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (2, 4), deadline = 20
Simulating trial. . .
epsilon = 0.1959; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1959; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1959; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 1.45610745089
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.4561074508884018, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.46)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 2.59850845653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.598508456529751, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.60)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: None, reward: 2.15181615725
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.1518161572496206, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.15)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 1.0521608517
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.0521608517044951, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.05)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 1.82975556912
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.8297555691207241, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.83)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 2.47300511334
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.473005113343449, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.47)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: 1.3922107468
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.3922107468044511, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent drove right instead of forward. (rewarded 1.39)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 1.03023632346
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.030236323460243, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent drove right instead of left. (rewarded 1.03)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 2.06049619078
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.0604961907838035, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.06)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: 0.182009958171
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 0.18200995817101018, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove left instead of right. (rewarded 0.18)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 0.1491349939
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'left'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.14913499390011442, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'left')
Agent drove right instead of left. (rewarded 0.15)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: -5.70100799376
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 9, 't': 11, 'action': None, 'reward': -5.70100799375879, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.70)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 1.7098804815
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.7098804815007917, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.71)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 0.971742852588
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 0.9717428525875864, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.97)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: -0.328337610435
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': -0.32833761043475096, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded -0.33)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.02829367246
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.0282936724607203, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.03)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.57500417376
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.5750041737636533, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.58)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: forward, reward: 0.744794958716
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.7447949587158218, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.74)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: 0.334621732664
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 0.3346217326636318, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.33)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: left, reward: -39.4941733302
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 1, 't': 19, 'action': 'left', 'reward': -39.49417333023549, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.49)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 164
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (2, 7), deadline = 20
Simulating trial. . .
epsilon = 0.1940; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1940; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: left, reward: 1.19771351753
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.1977135175325506, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'left')
Agent followed the waypoint left. (rewarded 1.20)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 1.28252916097
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.2825291609749463, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.28)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 0.48804314517
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.4880431451697186, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.49)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 1.29756557932
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.2975655793235932, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent drove right instead of left. (rewarded 1.30)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 1.55257699632
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.5525769963227114, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.55)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: left, reward: -9.41539893562
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', None, 'forward'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': -9.415398935621546, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.42)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: -0.0178811380257
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'right'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': -0.017881138025712007, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'right')
Agent drove forward instead of right. (rewarded -0.02)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: -9.4181815249
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -9.418181524898323, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.42)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 2.28588720761
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'right', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.2858872076119603, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', None)
Agent followed the waypoint right. (rewarded 2.29)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 2.22506394824
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', 'right'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 2.225063948240778, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', 'right')
Agent followed the waypoint right. (rewarded 2.23)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 1.86892724675
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.8689272467464997, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.87)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: 1.06030315207
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 1.0603031520709176, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.06)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 1.44821128424
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.4482112842408685, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 1.45)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.58798475302
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.5879847530190845, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 1.59)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 1.4596223895
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.4596223894950093, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.46)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: left, reward: 1.04525794294
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 5, 't': 15, 'action': 'left', 'reward': 1.045257942937459, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.05)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 165
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (3, 5), deadline = 20
Simulating trial. . .
epsilon = 0.1920; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.66241459043
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.6624145904263448, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 1.66)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 2.43042218988
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.430422189883338, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.43)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 2.68779809567
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.687798095673137, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.69)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 0.763143829207
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 0.7631438292069664, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove right instead of left. (rewarded 0.76)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.27661764151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.2766176415124857, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.28)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 2.30123627742
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.3012362774211135, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 2.30)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 0.898949046785
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.8989490467847303, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 0.90)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.78902827225
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.7890282722523927, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.79)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.34681950492
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.346819504915752, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.35)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.83875349865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.8387534986489722, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.84)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: forward, reward: 1.8509754619
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.8509754619039192, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.85)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.7891882961
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.7891882961048817, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.79)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.32898845911
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.328988459109643, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.33)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 0.651431210506
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 0.6514312105063673, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 0.65)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 2.17142841883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.1714284188300885, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.17)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.69629191665
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.6962919166492278, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.70)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: forward, reward: 2.23963333558
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 2.2396333355753013, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 2.24)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: 0.821766702463
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.8217667024630195, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.82)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: forward, reward: 0.642072779891
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 0.6420727798908585, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent drove forward instead of right. (rewarded 0.64)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: None, reward: 1.67870766922
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.678707669220627, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.68)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 166
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (6, 5), deadline = 30
Simulating trial. . .
epsilon = 0.1901; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 1.56256743577
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', 'right'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.5625674357672683, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', 'right')
Agent drove right instead of forward. (rewarded 1.56)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: 2.17849363308
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.1784936330779647, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.18)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: 1.27310008789
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.2731000878908312, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.27)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: 1.85042380632
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.850423806320006, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.85)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: None, reward: 1.82148672494
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.8214867249365303, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.82)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: left, reward: 1.28686904368
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 1.2868690436842096, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.29)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 1.42406854473
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.4240685447317927, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 1.42)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 2.66845957038
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 23, 't': 7, 'action': None, 'reward': 2.6684595703830434, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.67)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 1.00726143439
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.0072614343888124, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.01)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: 1.40414029365
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.404140293646188, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.40)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 2.67051769124
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.670517691241356, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.67)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.59539262967
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.595392629665068, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.60)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 2.15195368339
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.1519536833884425, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.15)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 0.43037861367
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 0.43037861366963703, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.43)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 1.00287900502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.0028790050176295, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent drove right instead of left. (rewarded 1.00)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: -40.0574497691
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': -40.05744976913077, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.06)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: -0.224262869686
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 14, 't': 16, 'action': 'right', 'reward': -0.22426286968562748, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent drove right instead of left. (rewarded -0.22)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 1.23937932985
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.2393793298545908, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.24)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: right, reward: 0.740938532057
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'forward'), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 0.7409385320569728, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'forward')
Agent followed the waypoint right. (rewarded 0.74)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.03292579206
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 11, 't': 19, 'action': None, 'reward': 1.032925792064788, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.03)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 1.49176827441
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 1.4917682744056666, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 1.49)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 0.764986168436
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 9, 't': 21, 'action': None, 'reward': 0.7649861684362977, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 0.76)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.61482305649
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 8, 't': 22, 'action': None, 'reward': 1.6148230564929689, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.61)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.01699255653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 7, 't': 23, 'action': None, 'reward': 1.0169925565345683, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.02)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.34190361142
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 6, 't': 24, 'action': None, 'reward': 1.34190361142485, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.34)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.33520460754
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 25, 'action': None, 'reward': 2.3352046075372, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.34)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: 0.722795841781
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 26, 'action': 'left', 'reward': 0.7227958417811349, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.72)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 1.96320609265
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 27, 'action': None, 'reward': 1.9632060926531103, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.96)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 1.20007537884
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 2, 't': 28, 'action': None, 'reward': 1.200075378842059, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.20)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 0.494964587121
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 1, 't': 29, 'action': 'forward', 'reward': 0.49496458712118274, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.49)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 167
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (7, 3), deadline = 25
Simulating trial. . .
epsilon = 0.1882; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1882; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1882; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.62520472455
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.6252047245454024, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.63)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.80182587532
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.8018258753226806, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.80)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 2.18251960286
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.182519602858176, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.18)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: forward, reward: 1.6363335799
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.6363335799001097, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded 1.64)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 0.978770296851
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 0.9787702968509251, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.98)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: -0.044698129789
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': -0.04469812978895249, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent drove forward instead of left. (rewarded -0.04)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 1.02923826779
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'forward'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.029238267793297, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.03)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 1.82543380938
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'forward'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.8254338093823301, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.83)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 2.41001929899
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.410019298991279, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.41)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 2.22545254575
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.2254525457490333, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.23)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: -10.7089853927
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': -10.7089853927285, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -10.71)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: forward, reward: 2.14962197445
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 2.1496219744498504, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 2.15)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: 1.62618183809
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.6261818380873068, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.63)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: 2.74182270232
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 2.7418227023214636, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.74)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: 1.14711072909
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.1471107290877862, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.15)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: None, reward: 2.06359781099
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.063597810991925, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.06)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: forward, reward: 1.2949312238
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 1.2949312238032185, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.29)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: None, reward: 1.00985283607
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 8, 't': 17, 'action': None, 'reward': 1.0098528360714287, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.01)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: forward, reward: -9.00990445371
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': -9.009904453713276, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.01)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: forward, reward: 1.19309182125
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 6, 't': 19, 'action': 'forward', 'reward': 1.1930918212478159, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.19)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: -4.61118969712
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, 'right', 'right'), 'deadline': 5, 't': 20, 'action': None, 'reward': -4.61118969712179, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.61)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 0.786140620573
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 0.7861406205728968, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent drove right instead of left. (rewarded 0.79)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: 0.0970565031534
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'left'), 'deadline': 3, 't': 22, 'action': 'left', 'reward': 0.09705650315341785, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'left')
Agent drove left instead of forward. (rewarded 0.10)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 1.34629125707
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 2, 't': 23, 'action': None, 'reward': 1.3462912570691092, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.35)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 0.434633961062
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.4346339610623666, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 0.43)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 168
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (2, 2), deadline = 25
Simulating trial. . .
epsilon = 0.1864; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 2.5218612762
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'left'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.521861276202161, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'left')
Agent followed the waypoint right. (rewarded 2.52)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 2.39236424064
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.392364240643614, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.39)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 2.14752343914
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.14752343913576, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.15)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: right, reward: 2.09023503946
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 2.0902350394616964, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.09)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 0.315923242934
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 0.31592324293427654, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent drove right instead of forward. (rewarded 0.32)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: None, reward: 2.83536369195
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.8353636919476934, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.84)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: left, reward: -39.5362301453
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': -39.53623014533871, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.54)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 0.015610574781
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.015610574781022235, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 0.02)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 2.68316931076
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.683169310756578, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.68)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 1.84592149509
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.8459214950932017, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.85)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: right, reward: 1.77875004494
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.7787500449394393, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.78)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: -40.4660303137
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'forward'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': -40.46603031370504, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.47)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: -10.5713243362
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': -10.571324336192886, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -10.57)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: forward, reward: 2.68431929329
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 2.6843192932945015, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.68)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: 2.16005636184
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 2.1600563618433073, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 2.16)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: left, reward: 2.63654587602
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 10, 't': 15, 'action': 'left', 'reward': 2.6365458760235274, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 2.64)
36% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 169
\-------------------------
Environment.reset(): Trial set up with start = (5, 2), destination = (6, 5), deadline = 20
Simulating trial. . .
epsilon = 0.1845; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 2.45579086858
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.4557908685753516, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 2.46)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: left, reward: -10.9978320105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -10.997832010533715, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -11.00)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 2.19442919843
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.194429198428751, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.19)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 1.17621099957
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.17621099957455, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.18)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: right, reward: 2.58568336219
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.5856833621915274, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 2.59)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 0.942811133854
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 0.9428111338536247, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 0.94)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 0.537336014675
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.5373360146749473, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 0.54)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: -40.5318330023
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'right'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': -40.531833002301624, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.53)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 1.53021226788
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'right'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.5302122678820085, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 1.53)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: 0.720969759387
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 0.7209697593869937, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 0.72)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: 0.630347491517
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.630347491516749, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent drove right instead of left. (rewarded 0.63)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 0.929350213278
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.9293502132777096, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.93)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 1.08732482022
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.0873248202154098, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.09)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: left, reward: 2.02992103334
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 2.0299210333360094, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.03)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: forward, reward: 1.75047979557
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 1.7504797955688722, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.75)
25% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 170
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (3, 6), deadline = 20
Simulating trial. . .
epsilon = 0.1827; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.11324897394
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.1132489739389144, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.11)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 2.65012529832
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.6501252983209973, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.65)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: forward, reward: -10.6528281023
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': -10.652828102262536, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.65)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 2.28604964812
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.2860496481182198, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.29)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: 2.91788195301
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.9178819530093927, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.92)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: -4.98572153342
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': -4.985721533416093, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.99)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: 1.17657506926
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.176575069256086, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.18)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: left, reward: 0.959981044994
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 0.9599810449937569, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 0.96)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: forward, reward: -10.9458850442
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', 'forward'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -10.945885044214265, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -10.95)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: left, reward: 0.786754826642
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 0.7867548266417177, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.79)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: left, reward: 1.10194727771
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 1.1019472777085983, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove left instead of right. (rewarded 1.10)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 1.5835711396
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.5835711395982277, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.58)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: 0.536957997836
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.5369579978362284, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent drove right instead of left. (rewarded 0.54)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 1.10047929009
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.1004792900914875, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.10)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 0.929601244232
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.9296012442321093, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.93)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: None, reward: 1.26332024914
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.263320249140823, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.26)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: right, reward: 2.07854395017
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 2.0785439501723957, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.08)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: right, reward: 0.842466647902
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.8424666479022438, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 0.84)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: 1.28079874253
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 1.2807987425298948, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent followed the waypoint forward. (rewarded 1.28)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: -40.7581035851
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': -40.758103585058954, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.76)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 171
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (3, 7), deadline = 30
Simulating trial. . .
epsilon = 0.1809; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1809; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1809; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 1.99438425499
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.9943842549851933, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.99)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 2.20732288173
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.20732288173207, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.21)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: None, reward: 2.65674824745
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', 'forward'), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.6567482474493715, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.66)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: 0.323409780901
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 0.323409780900738, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.32)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: left, reward: 0.994371984447
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 0.9943719844474312, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.99)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 1.4545444256
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 25, 't': 5, 'action': None, 'reward': 1.4545444256037166, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.45)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: None, reward: 2.78801381342
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.788013813419158, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.79)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: 1.8025787582
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 1.8025787581972568, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.80)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: forward, reward: 2.16985899925
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': 2.1698589992485613, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.17)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 1.19774297193
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 1.1977429719251464, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.20)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: forward, reward: 2.73469446077
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 2.734694460773198, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.73)
63% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 172
\-------------------------
Environment.reset(): Trial set up with start = (7, 3), destination = (3, 2), deadline = 25
Simulating trial. . .
epsilon = 0.1791; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: left, reward: 2.77446930591
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 2.774469305909398, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.77)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 2.5761775028
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.576177502797904, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.58)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 1.06152025401
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.0615202540124014, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.06)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 1.83174109075
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.8317410907530263, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.83)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 1.10586769803
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 1.1058676980256048, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.11)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: left, reward: -9.614680645
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': -9.614680644995891, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent attempted driving left through a red light. (rewarded -9.61)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.60891397865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.608913978649019, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.61)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 2.39797811436
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.3979781143635215, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.40)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 0.313571719479
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.31357171947899987, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 0.31)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: left, reward: 1.04385494093
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.0438549409327802, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.04)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 0.942304787744
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 15, 't': 10, 'action': None, 'reward': 0.9423047877437707, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.94)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 2.43197831695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.4319783169522315, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.43)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 1.70693247662
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.706932476617357, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.71)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: left, reward: 1.91480217171
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 1.9148021717091699, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.91)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: right, reward: 0.43309453419
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 0.43309453418997, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', None)
Agent drove right instead of forward. (rewarded 0.43)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 2.42703015726
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.4270301572572257, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.43)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: 1.58194044662
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 1.5819404466211862, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent drove forward instead of left. (rewarded 1.58)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: right, reward: -0.0143255121913
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': -0.014325512191256884, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded -0.01)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: left, reward: -0.426888184062
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', None), 'deadline': 7, 't': 18, 'action': 'left', 'reward': -0.42688818406198137, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', None)
Agent drove left instead of right. (rewarded -0.43)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 0.550115952351
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 6, 't': 19, 'action': None, 'reward': 0.5501159523510886, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 0.55)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: right, reward: 0.0736273956151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 0.07362739561507747, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent drove right instead of left. (rewarded 0.07)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 1.90438806016
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 1.9043880601616174, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.90)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 0.270982562265
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 3, 't': 22, 'action': None, 'reward': 0.2709825622648727, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.27)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 0.819982673811
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 2, 't': 23, 'action': None, 'reward': 0.8199826738108209, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 0.82)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 1.72424538916
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 1, 't': 24, 'action': None, 'reward': 1.7242453891561276, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.72)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 173
\-------------------------
Environment.reset(): Trial set up with start = (2, 6), destination = (8, 3), deadline = 25
Simulating trial. . .
epsilon = 0.1773; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1773; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1773; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1773; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: left, reward: 2.007642474
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 2.0076424740025693, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.01)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.64886663962
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.6488666396233431, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.65)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 0.257847578468
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 0.25784757846756556, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.26)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: 1.95098577713
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.9509857771349277, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent drove forward instead of left. (rewarded 1.95)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: left, reward: 1.13222052535
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 1.1322205253503412, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.13)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 2.80668102032
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.806681020316476, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.81)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 1.73890125944
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.7389012594445419, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.74)
72% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 174
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (8, 2), deadline = 25
Simulating trial. . .
epsilon = 0.1755; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 1.88354547659
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.8835454765882491, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.88)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 1.95598375318
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.9559837531778732, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.96)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 1.90737646996
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.9073764699633569, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.91)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 1.12973587379
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.1297358737935692, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.13)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.3047651457
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.304765145697332, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.30)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: left, reward: 2.46104022476
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 2.4610402247636864, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.46)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: forward, reward: 1.82955263295
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.829552632949131, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 1.83)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: 1.72574257925
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.7257425792481322, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.73)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: 2.34399438713
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.343994387128623, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.34)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: left, reward: 1.83313605137
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.8331360513728028, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.83)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.30290355459
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.3029035545868215, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.30)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.22644223022
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.2264422302249134, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.23)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.09092488721
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.0909248872127215, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.09)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: forward, reward: 1.81528162667
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 1.8152816266717478, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.82)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.33283098588
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.3328309858757112, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.33)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: 0.236438995177
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 0.23643899517742772, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent drove right instead of forward. (rewarded 0.24)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: -0.0998704032214
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': -0.09987040322135277, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded -0.10)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 0.621807439444
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 0.621807439444384, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.62)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: 1.58224751201
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 1.5822475120092057, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.58)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: left, reward: 0.101635955372
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 0.10163595537226577, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.10)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 1.07120639973
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.0712063997265555, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.07)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 0.824856382089
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 0.8248563820889556, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.82)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 1.2072053872
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 1.2072053872029933, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.21)
8% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 175
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (8, 7), deadline = 35
Simulating trial. . .
epsilon = 0.1738; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1738; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1738; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1738; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: 2.09576851358
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'left'), 'deadline': 35, 't': 0, 'action': None, 'reward': 2.095768513575437, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.10)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 1.24240302343
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 34, 't': 1, 'action': 'right', 'reward': 1.2424030234271173, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent drove right instead of forward. (rewarded 1.24)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: forward, reward: -39.730232373
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 33, 't': 2, 'action': 'forward', 'reward': -39.730232372957985, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.73)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: forward, reward: -9.08635858459
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 32, 't': 3, 'action': 'forward', 'reward': -9.086358584587712, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent attempted driving forward through a red light. (rewarded -9.09)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: left, reward: 2.59688221607
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 31, 't': 4, 'action': 'left', 'reward': 2.5968822160697376, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.60)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: None, reward: 1.66573565907
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 30, 't': 5, 'action': None, 'reward': 1.6657356590746637, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.67)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: forward, reward: 1.48161151434
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 29, 't': 6, 'action': 'forward', 'reward': 1.481611514337708, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.48)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: None, reward: 0.997466937041
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 7, 'action': None, 'reward': 0.9974669370408826, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.00)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: -10.770191646
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 8, 'action': 'left', 'reward': -10.770191646035283, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.77)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 1.94469718297
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 26, 't': 9, 'action': 'forward', 'reward': 1.9446971829701822, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 1.94)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.5528015316
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 25, 't': 10, 'action': None, 'reward': 2.552801531601188, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.55)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 0.95363563297
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'right'), 'deadline': 24, 't': 11, 'action': 'right', 'reward': 0.9536356329702661, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'right')
Agent drove right instead of forward. (rewarded 0.95)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 0.85228475387
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 23, 't': 12, 'action': 'right', 'reward': 0.8522847538698489, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 0.85)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 0.868428768114
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 22, 't': 13, 'action': 'left', 'reward': 0.868428768114283, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 0.87)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 0.954257347137
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'left'), 'deadline': 21, 't': 14, 'action': None, 'reward': 0.9542573471369145, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 0.95)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 2.12975952003
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 20, 't': 15, 'action': None, 'reward': 2.1297595200262154, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.13)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.78097854152
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 16, 'action': None, 'reward': 1.7809785415158126, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.78)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: left, reward: 2.71117652605
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 18, 't': 17, 'action': 'left', 'reward': 2.7111765260533716, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.71)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: right, reward: -0.251494432039
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', 'left'), 'deadline': 17, 't': 18, 'action': 'right', 'reward': -0.25149443203923005, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', 'left')
Agent drove right instead of forward. (rewarded -0.25)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: left, reward: 0.833710448293
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 19, 'action': 'left', 'reward': 0.8337104482930626, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.83)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.88083939959
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 15, 't': 20, 'action': None, 'reward': 1.8808393995896058, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.88)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 0.865298983509
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'right'), 'deadline': 14, 't': 21, 'action': 'right', 'reward': 0.8652989835094567, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'right')
Agent drove right instead of left. (rewarded 0.87)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 2.2935141928
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', 'forward'), 'deadline': 13, 't': 22, 'action': 'right', 'reward': 2.2935141927952323, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', 'forward')
Agent followed the waypoint right. (rewarded 2.29)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 0.756167544762
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 12, 't': 23, 'action': 'right', 'reward': 0.7561675447615641, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.76)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 1.17034428333
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 11, 't': 24, 'action': 'right', 'reward': 1.1703442833341149, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.17)
29% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 0.766261545457
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 10, 't': 25, 'action': None, 'reward': 0.7662615454573687, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 0.77)
26% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.78256333948
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 9, 't': 26, 'action': None, 'reward': 1.7825633394825053, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.78)
23% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 2.00006903244
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'right'), 'deadline': 8, 't': 27, 'action': None, 'reward': 2.0000690324356913, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'right')
Agent properly idled at a red light. (rewarded 2.00)
20% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 1.19803762254
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'forward'), 'deadline': 7, 't': 28, 'action': 'forward', 'reward': 1.1980376225421117, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.20)
17% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 1.18178950832
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 6, 't': 29, 'action': 'right', 'reward': 1.1817895083227197, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent drove right instead of left. (rewarded 1.18)
14% of time remaining to reach destination.
/-------------------
| Step 30 Results
\-------------------
Environment.step(): t = 30
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.78053764795
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 5, 't': 30, 'action': 'right', 'reward': 1.7805376479524195, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.78)
11% of time remaining to reach destination.
/-------------------
| Step 31 Results
\-------------------
Environment.step(): t = 31
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 0.361604017857
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 4, 't': 31, 'action': 'right', 'reward': 0.36160401785725327, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 0.36)
9% of time remaining to reach destination.
/-------------------
| Step 32 Results
\-------------------
Environment.step(): t = 32
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 1.00308568211
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', 'forward'), 'deadline': 3, 't': 32, 'action': 'right', 'reward': 1.0030856821061147, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', 'forward')
Agent drove right instead of forward. (rewarded 1.00)
6% of time remaining to reach destination.
/-------------------
| Step 33 Results
\-------------------
Environment.step(): t = 33
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 2.07897412387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 2, 't': 33, 'action': None, 'reward': 2.0789741238723325, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.08)
3% of time remaining to reach destination.
/-------------------
| Step 34 Results
\-------------------
Environment.step(): t = 34
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: 0.321537876363
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 1, 't': 34, 'action': 'left', 'reward': 0.3215378763625034, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.32)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 176
\-------------------------
Environment.reset(): Trial set up with start = (4, 3), destination = (6, 5), deadline = 20
Simulating trial. . .
epsilon = 0.1720; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1720; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 2.52900352677
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.529003526770767, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.53)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: -40.9257616482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', 'left'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': -40.925761648178074, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.93)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 2.27236086813
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.2723608681341716, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.27)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 2.19208458406
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.1920845840581045, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.19)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: 2.68646590513
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.6864659051322164, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.69)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: left, reward: 1.33205354519
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.3320535451873325, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent drove left instead of forward. (rewarded 1.33)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: right, reward: 1.65326040512
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.653260405117428, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.65)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 0.0783402596884
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.07834025968835001, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.08)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: forward, reward: 1.59804356354
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.5980435635427175, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.60)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 1.62467464776
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', 'left'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.6246746477570522, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', 'left')
Agent drove right instead of forward. (rewarded 1.62)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 2.0167571902
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.0167571902043986, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.02)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.71988716796
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.719887167961497, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.72)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 0.788013977981
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.7880139779808988, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.79)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: left, reward: 1.17799661654
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.177996616542916, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.18)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 2.06522857765
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.065228577645498, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.07)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 1.99076558491
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.99076558490852, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.99)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: left, reward: 1.44891454038
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 1.4489145403831603, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.45)
15% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 177
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (6, 7), deadline = 25
Simulating trial. . .
epsilon = 0.1703; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1703; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: left, reward: 2.67710241528
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 2.677102415281979, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.68)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: left, reward: 1.33409109375
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 1.3340910937464117, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.33)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: right, reward: 2.59105295324
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 2.591052953239159, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.59)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 2.58681619716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.586816197161962, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.59)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 2.02594581776
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.02594581775583, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.03)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: left, reward: 2.02690382972
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 2.0269038297242203, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 2.03)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: None, reward: 2.09325901375
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.0932590137512195, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.09)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: forward, reward: 2.36066189842
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 2.3606618984202683, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.36)
68% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 178
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (7, 2), deadline = 20
Simulating trial. . .
epsilon = 0.1686; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1686; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 1.40916832822
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.409168328223966, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.41)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 0.268366246676
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.2683662466761455, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.27)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 1.79327487974
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.7932748797440181, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.79)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 1.63688511545
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.63688511545211, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.64)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: left, reward: 1.91729705016
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.917297050162816, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 1.92)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 2.06072901833
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.0607290183326104, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.06)
70% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 179
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (1, 7), deadline = 20
Simulating trial. . .
epsilon = 0.1670; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: None, reward: -5.87004233
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'right'), 'deadline': 20, 't': 0, 'action': None, 'reward': -5.870042329995232, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.87)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: left, reward: 1.86106851375
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'left'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.861068513750448, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'left')
Agent followed the waypoint left. (rewarded 1.86)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: left, reward: 1.83296029549
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.8329602954913993, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove left instead of forward. (rewarded 1.83)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 1.46583374802
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.4658337480236068, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.47)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 1.58903482594
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.5890348259368592, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.59)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 1.5891150224
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.5891150223973003, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.59)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 1.39732274929
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'right'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.3973227492851947, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'right')
Agent drove right instead of forward. (rewarded 1.40)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 2.48348236541
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 2.483482365410996, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.48)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 0.753920641372
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.7539206413722473, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.75)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: 0.462508772576
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 0.462508772576395, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent drove forward instead of right. (rewarded 0.46)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 1.76311075625
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.7631107562466601, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.76)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: forward, reward: -9.5241853618
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': -9.524185361803141, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.52)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.51481103431
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.5148110343092445, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.51)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: 1.64494945256
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.6449494525642774, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.64)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 0.981400897378
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 0.9814008973778743, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 0.98)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: forward, reward: 1.47717425166
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 1.4771742516555644, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.48)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 1.91046265681
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.9104626568101708, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.91)
15% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 180
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (4, 3), deadline = 20
Simulating trial. . .
epsilon = 0.1653; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1653; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: None, reward: 2.65503544842
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.655035448418722, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.66)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: right, reward: 0.0367402435961
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'right', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.03674024359609096, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'right', None)
Agent drove right instead of left. (rewarded 0.04)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 2.79700741045
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.797007410445791, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 2.80)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 1.09508254677
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.09508254677223, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.10)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.35310449083
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.3531044908308825, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.35)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.07477225292
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.0747722529192365, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.07)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.06127186702
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.0612718670228363, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.06)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 1.93631727416
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.9363172741623214, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent followed the waypoint forward. (rewarded 1.94)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.1621275882
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.1621275881988387, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.16)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 2.61364995473
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 2.6136499547294463, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.61)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 0.77379956756
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 0.773799567559601, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 0.77)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 181
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (8, 6), deadline = 30
Simulating trial. . .
epsilon = 0.1637; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: 2.85312454459
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': 2.8531245445898956, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.85)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 2.68883519513
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.6888351951334597, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 2.69)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: -9.34806430519
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': -9.348064305188366, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.35)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 2.17870820051
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.178708200511777, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.18)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 2.19538928164
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.1953892816421394, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.20)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 1.71745794188
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.7174579418799625, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.72)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 1.48593369671
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 1.4859336967066739, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.49)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.42336907977
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 2.4233690797749343, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.42)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.92386148987
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.9238614898740933, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.92)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.46424104226
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 21, 't': 9, 'action': None, 'reward': 1.4642410422580012, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.46)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 0.688123831595
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 0.6881238315946931, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent drove right instead of forward. (rewarded 0.69)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 2.11433836757
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.114338367569075, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.11)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 2.25248117783
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.252481177825583, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.25)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: left, reward: 1.03429640693
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 1.034296406926031, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.03)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 1.77004950424
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': 1.7700495042378357, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent drove forward instead of right. (rewarded 1.77)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 2.12032172558
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 15, 't': 15, 'action': 'right', 'reward': 2.1203217255770666, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 2.12)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: left, reward: -39.9595086364
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 14, 't': 16, 'action': 'left', 'reward': -39.95950863643686, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.96)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 0.795683142708
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 13, 't': 17, 'action': None, 'reward': 0.7956831427077529, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.80)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 2.37816577413
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', 'left'), 'deadline': 12, 't': 18, 'action': None, 'reward': 2.378165774133227, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.38)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 1.39329211909
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', 'left'), 'deadline': 11, 't': 19, 'action': None, 'reward': 1.393292119090546, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', 'left')
Agent properly idled at a red light. (rewarded 1.39)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 1.29746020504
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 1.2974602050363784, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.30)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: right, reward: -0.130373967972
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 9, 't': 21, 'action': 'right', 'reward': -0.13037396797206247, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent drove right instead of left. (rewarded -0.13)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: None, reward: 2.06224866402
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 22, 'action': None, 'reward': 2.0622486640168782, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.06)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 0.715854505848
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 0.7158545058477224, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.72)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 0.8160376365
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 0.8160376365001165, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', None)
Agent followed the waypoint right. (rewarded 0.82)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 1.96843276711
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 25, 'action': 'right', 'reward': 1.968432767113101, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.97)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: 0.669066658995
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 26, 'action': 'left', 'reward': 0.669066658994887, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.67)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 0.503956014828
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 3, 't': 27, 'action': 'right', 'reward': 0.5039560148281925, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.50)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: left, reward: 1.43712593229
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 2, 't': 28, 'action': 'left', 'reward': 1.4371259322866705, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.44)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 0.804812489475
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 1, 't': 29, 'action': None, 'reward': 0.8048124894752107, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 0.80)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 182
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (3, 3), deadline = 25
Simulating trial. . .
epsilon = 0.1620; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1620; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 1.46888902552
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.468889025518206, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.47)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: 2.29504843765
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 2.295048437645461, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 2.30)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: left, reward: 1.68883007785
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 1.6888300778501262, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent drove left instead of forward. (rewarded 1.69)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 1.20472011176
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.2047201117576174, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.20)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 0.230529955466
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 0.2305299554657393, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'left')
Agent drove right instead of left. (rewarded 0.23)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 2.75069765839
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 2.750697658385281, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.75)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: right, reward: 2.75832454247
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.7583245424653757, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent followed the waypoint right. (rewarded 2.76)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 2.51230281877
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 2.5123028187742857, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 2.51)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: None, reward: 1.3079872202
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.307987220195251, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.31)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 0.269607898543
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.26960789854257716, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded 0.27)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: left, reward: -10.5319391781
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, 'forward'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': -10.531939178077193, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.53)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 2.45621799527
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'forward'), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.456217995267406, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.46)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: forward, reward: -9.74484501735
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', 'right', 'forward'), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': -9.744845017348014, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.74)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 1.46983698909
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.4698369890926566, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.47)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: right, reward: 1.87869826648
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 1.878698266484552, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.88)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: right, reward: 1.76013922412
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'right'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 1.7601392241173555, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 1.76)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: forward, reward: -9.93650921849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': -9.936509218487254, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.94)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: left, reward: 2.36425328664
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 2.3642532866386685, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.36)
28% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 183
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (8, 2), deadline = 30
Simulating trial. . .
epsilon = 0.1604; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: right, reward: 1.88177295428
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.8817729542832617, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove right instead of left. (rewarded 1.88)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 2.83256177612
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.8325617761178967, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.83)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 1.7768541138
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.7768541137991156, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.78)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: left, reward: -9.42982154322
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'left', 'reward': -9.429821543216772, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -9.43)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 1.74457058171
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.7445705817139814, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.74)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: 1.83686888928
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 1.8368688892782594, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.84)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 2.64659148864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 2.646591488640446, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.65)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: 2.86781712862
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 2.8678171286168466, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent followed the waypoint forward. (rewarded 2.87)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: 2.26843097042
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 2.26843097042285, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.27)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: forward, reward: 1.849112215
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.849112214997966, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.85)
67% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 184
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (2, 5), deadline = 30
Simulating trial. . .
epsilon = 0.1588; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 2.69579771775
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 2.6957977177514674, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.70)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.43541807512
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.4354180751186116, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.44)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.16627675845
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.1662767584469496, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.17)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 2.94031497383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 2.940314973826621, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.94)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 2.48700460899
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.4870046089886286, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.49)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: None, reward: 2.55651217376
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', 'left'), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.5565121737614005, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.56)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: 1.40474202129
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 24, 't': 6, 'action': 'left', 'reward': 1.404742021293567, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.40)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 1.87546746692
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 1.8754674669158025, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.88)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: left, reward: -40.3023944395
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': -40.30239443945739, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.30)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 0.983050725492
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 0.9830507254918217, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 0.98)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 1.84339981819
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 1.843399818194095, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.84)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 2.00659929325
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 11, 'action': None, 'reward': 2.0065992932522225, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.01)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 2.23820221115
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 18, 't': 12, 'action': None, 'reward': 2.2382022111461883, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.24)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: 2.50015620206
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 17, 't': 13, 'action': 'left', 'reward': 2.500156202060929, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.50)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: 2.42950788613
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 16, 't': 14, 'action': 'left', 'reward': 2.429507886126686, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 2.43)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 185
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (4, 4), deadline = 20
Simulating trial. . .
epsilon = 0.1572; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 2.18438719229
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.184387192290812, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.18)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.14383377036
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.1438337703636159, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.14)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 2.0038972386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.0038972385981735, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.00)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: left, reward: 1.44631492792
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 1.4463149279191918, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.45)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 0.98095898954
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 0.9809589895398487, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.98)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: left, reward: 1.41904647616
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.4190464761585704, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.42)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: left, reward: 0.217080370802
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 0.21708037080247422, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 0.22)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: 1.8991407843
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.899140784297815, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 1.90)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: forward, reward: -9.61924812157
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -9.619248121571053, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving forward through a red light. (rewarded -9.62)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: right, reward: 2.6327723173
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 2.6327723173049478, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 2.63)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: -4.9803028399
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 10, 't': 10, 'action': None, 'reward': -4.9803028398978455, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.98)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 1.64437076346
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.6443707634599904, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.64)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 1.29173379591
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.291733795910885, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.29)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 0.733309834091
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 0.7333098340913331, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.73)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 1.71233941235
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.7123394123485316, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.71)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: left, reward: -10.0567567874
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -10.056756787371116, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.06)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: 1.08683508021
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.0868350802135327, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.09)
15% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 186
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (2, 7), deadline = 30
Simulating trial. . .
epsilon = 0.1557; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: 2.8588202927
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': 2.85882029270196, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.86)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 1.64921279912
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.6492127991186694, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove right instead of forward. (rewarded 1.65)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: forward, reward: 0.515476953021
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 0.5154769530214754, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove forward instead of left. (rewarded 0.52)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: left, reward: 2.61139478017
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 27, 't': 3, 'action': 'left', 'reward': 2.611394780173168, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.61)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 2.55053916405
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.550539164045374, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.55)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 2.04296778014
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 2.042967780139582, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.04)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 2.83708774954
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 2.8370877495383873, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent followed the waypoint right. (rewarded 2.84)
77% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 187
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (1, 5), deadline = 20
Simulating trial. . .
epsilon = 0.1541; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: 0.0917724275742
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.09177242757418114, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.09)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.84763945784
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.847639457838393, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.85)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.50559205502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.505592055024326, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.51)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.60293473567
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.6029347356662598, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.60)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 1.32703431622
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.3270343162188176, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.33)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: left, reward: -39.5903929464
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': -39.59039294637081, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.59)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 1.55483640002
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.554836400019136, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 1.55)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: 1.24308409726
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.2430840972620052, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.24)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 2.51425452587
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 2.5142545258699256, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 2.51)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.2152816627
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.215281662697502, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 1.22)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 1.24191920888
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.241919208880224, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.24)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: -19.367099567
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', 'right'), 'deadline': 9, 't': 11, 'action': 'right', 'reward': -19.367099567006107, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', 'right')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.37)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.95966602676
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.9596660267553612, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.96)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.05039210096
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.0503921009595043, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.05)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 1.56375678823
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.563756788231654, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 1.56)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 1.4578580574
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.4578580574049602, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.46)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 0.495901854812
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.4959018548117171, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 0.50)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: left, reward: 1.57682428486
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 1.5768242848601766, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 1.58)
10% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 188
\-------------------------
Environment.reset(): Trial set up with start = (7, 5), destination = (5, 3), deadline = 20
Simulating trial. . .
epsilon = 0.1526; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 1.89475913271
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.894759132713656, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.89)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 2.44384867761
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.443848677606303, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 2.44)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 1.20024752173
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.2002475217285973, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.20)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: right, reward: 1.27380836477
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.2738083647661833, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.27)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 0.751209696705
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.7512096967049108, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', None)
Agent drove right instead of forward. (rewarded 0.75)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 1.41604810245
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.4160481024525928, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.42)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 0.419329462395
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 0.41932946239483737, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent drove forward instead of left. (rewarded 0.42)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: -10.5679278939
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -10.56792789389848, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.57)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.63835177357
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.6383517735658772, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.64)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: left, reward: 2.01523055042
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 2.015230550416537, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.02)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: right, reward: 0.79039132191
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.7903913219097399, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent drove right instead of left. (rewarded 0.79)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: forward, reward: 0.690470259292
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'right'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.690470259291833, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'right')
Agent drove forward instead of left. (rewarded 0.69)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: left, reward: 0.0953209076949
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 8, 't': 12, 'action': 'left', 'reward': 0.09532090769486135, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 0.10)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 1.19661793653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.1966179365313916, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.20)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: 1.14525393122
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.1452539312231538, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.15)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 0.677369892614
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.6773698926144254, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.68)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 1.19407478688
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.194074786877799, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 1.19)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 0.694676158887
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.6946761588871415, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.69)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 1.89824718824
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.89824718824279, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.90)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: 1.76746250653
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 1.767462506526725, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.77)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 189
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (5, 5), deadline = 25
Simulating trial. . .
epsilon = 0.1511; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1511; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1511; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1511; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1511; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1511; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: right, reward: 1.32718039665
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.3271803966461317, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.33)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 2.19319654334
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 2.1931965433447793, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent followed the waypoint forward. (rewarded 2.19)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: 1.27035006109
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 1.2703500610866596, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.27)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: right, reward: 2.84951638623
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 2.849516386230631, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 2.85)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: forward, reward: -9.62317337413
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -9.623173374132039, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.62)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 2.0077485767
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.007748576702678, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.01)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: 2.30262909052
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.3026290905200755, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.30)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 4), heading: (0, 1), action: None, reward: -5.48536738693
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': -5.485367386928011, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.49)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: forward, reward: 1.59802056044
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 1.5980205604430913, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.60)
64% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 190
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (6, 5), deadline = 25
Simulating trial. . .
epsilon = 0.1496; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1496; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: 2.13756558953
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.13756558953195, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 2.14)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 2.9343966114
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 2.9343966113967284, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.93)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 0.802119935849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 0.8021199358487403, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent drove right instead of left. (rewarded 0.80)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.58265303314
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.5826530331449664, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.58)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: forward, reward: 2.27509188608
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.275091886076476, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.28)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 0.942192713929
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 0.9421927139290631, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.94)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 2.3432857761
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.343285776104619, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.34)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: forward, reward: 2.226242504
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 2.2262425040039022, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 2.23)
68% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 191
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (5, 3), deadline = 20
Simulating trial. . .
epsilon = 0.1481; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 0.375270626971
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.37527062697057323, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 0.38)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: left, reward: 1.52463571685
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'left'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.5246357168509301, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'left')
Agent drove left instead of right. (rewarded 1.52)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 0.216931995149
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.21693199514904793, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'forward')
Agent drove right instead of left. (rewarded 0.22)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 2.28624138649
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.2862413864875393, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.29)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 0.0522745856808
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.05227458568080412, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.05)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: left, reward: 1.09426313884
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.0942631388418258, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.09)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 1.94621391713
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.946213917132222, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.95)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.14941473795
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.149414737954395, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.15)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.02794057947
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.0279405794683347, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.03)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: 1.99345342664
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.9934534266399984, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.99)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: right, reward: 0.948934565185
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'right', 'forward'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.9489345651847924, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.95)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: forward, reward: -10.2515086906
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': -10.251508690593946, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.25)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: None, reward: 2.16947982778
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.169479827780666, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.17)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: right, reward: 0.125422918166
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'forward'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.125422918166484, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'forward')
Agent drove right instead of left. (rewarded 0.13)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: left, reward: 2.21628567721
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 2.2162856772098176, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.22)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: -19.4266057854
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': -19.426605785353967, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.43)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: left, reward: 1.46513716806
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 1.4651371680582967, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 1.47)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 0.594828282226
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 0.594828282226368, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 0.59)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 0.236866915466
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.23686691546553562, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.24)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: forward, reward: 0.459171199119
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 0.4591711991187635, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 0.46)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 192
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (6, 4), deadline = 25
Simulating trial. . .
epsilon = 0.1466; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1466; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1466; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1466; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1466; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 0.365648841646
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 0.3656488416460715, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'forward')
Agent drove right instead of left. (rewarded 0.37)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.81219183962
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.812191839623541, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.81)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.41424377502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.4142437750216545, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.41)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 2.20924130069
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.2092413006888245, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.21)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: None, reward: 1.17585896509
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.1758589650868814, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.18)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 1.06185739305
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.0618573930457533, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.06)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 0.0985217043048
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.09852170430484486, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove right instead of forward. (rewarded 0.10)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: left, reward: 1.03642391858
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 1.0364239185805897, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.04)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 2.79487633931
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.7948763393082707, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.79)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: 2.63933284584
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.639332845835597, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.64)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 2.27452654056
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.274526540561209, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.27)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 0.697512766285
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 0.6975127662847028, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove right instead of left. (rewarded 0.70)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 1.43925267307
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.4392526730713258, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove right instead of forward. (rewarded 1.44)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: 1.16335785841
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 1.1633578584139543, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.16)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: 2.02854458564
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.0285445856441138, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.03)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: 1.25449978479
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.254499784791656, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.25)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: left, reward: 0.653499692834
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 0.6534996928342396, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 0.65)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 2.03347051112
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'forward'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 2.03347051112181, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.03)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: forward, reward: 2.20396680804
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': 2.203966808041593, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.20)
24% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 193
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (7, 5), deadline = 25
Simulating trial. . .
epsilon = 0.1451; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 1.26119267202
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.2611926720189321, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.26)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: left, reward: 0.793666538264
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 0.7936665382644088, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove left instead of forward. (rewarded 0.79)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 4), heading: (0, 1), action: left, reward: -19.1437147594
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'forward'), 'deadline': 23, 't': 2, 'action': 'left', 'reward': -19.143714759430924, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.14)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: forward, reward: 0.894742028231
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 0.894742028231355, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent drove forward instead of right. (rewarded 0.89)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: forward, reward: 1.6367160077
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 1.636716007703796, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 1.64)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 2.04225795148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.0422579514828225, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.04)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 0.965134208049
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.9651342080492571, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.97)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.31161300199
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.3116130019922818, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.31)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 2.62240457582
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.622404575822588, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.62)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 2.73648995854
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.7364899585393636, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.74)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 2.52312908026
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 2.5231290802569823, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 2.52)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 1.71588124767
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.7158812476650562, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.72)
52% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 194
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (2, 5), deadline = 25
Simulating trial. . .
epsilon = 0.1437; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: 2.3932584486
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.393258448595972, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 2.39)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 2.8338744231
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 2.8338744231022464, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.83)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 2.95785467211
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.9578546721060146, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.96)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: left, reward: -10.3252505878
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'left', 'reward': -10.32525058780936, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.33)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 1.46663908386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.4666390838615957, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.47)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 2.34621153942
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 2.3462115394246723, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.35)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: -10.8493921524
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -10.849392152398204, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.85)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.88253870807
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.8825387080685314, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.88)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 2.5073257045
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'forward'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.5073257045027546, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.51)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: forward, reward: 1.41696490819
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.4169649081948064, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.42)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: forward, reward: 1.65122393207
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 1.6512239320665696, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.65)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 195
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (7, 7), deadline = 20
Simulating trial. . .
epsilon = 0.1423; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: 2.0935190333
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'right'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.0935190332951628, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'right')
Agent followed the waypoint forward. (rewarded 2.09)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 1.1636412613
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.1636412613042808, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.16)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 2.94153657295
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.9415365729529364, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.94)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 0.991186990331
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 0.9911869903305701, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 0.99)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: forward, reward: 2.30690245523
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.3069024552289994, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent followed the waypoint forward. (rewarded 2.31)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: 0.0267876087435
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 0.026787608743512825, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.03)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: right, reward: 0.138333057598
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.1383330575975541, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.14)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 1.05699344076
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.0569934407634975, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.06)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 2.43422657806
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.434226578056988, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 2.43)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: 1.19649309864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.196493098635869, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove left instead of forward. (rewarded 1.20)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 0.924318859753
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.9243188597529897, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.92)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: forward, reward: 1.55889312179
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.558893121789213, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 1.56)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 1.70999529797
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.70999529796856, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.71)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 0.930411630646
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.9304116306457675, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.93)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 196
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (2, 7), deadline = 35
Simulating trial. . .
epsilon = 0.1409; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: forward, reward: 1.72332137621
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'forward'), 'deadline': 35, 't': 0, 'action': 'forward', 'reward': 1.7233213762080877, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'forward')
Agent drove forward instead of left. (rewarded 1.72)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: 1.13663405836
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 34, 't': 1, 'action': None, 'reward': 1.1366340583647367, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.14)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 0.611558905534
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'right'), 'deadline': 33, 't': 2, 'action': 'right', 'reward': 0.6115589055338566, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'right')
Agent drove right instead of left. (rewarded 0.61)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 2.87667282757
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 32, 't': 3, 'action': None, 'reward': 2.87667282756572, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.88)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 1.22654995629
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 31, 't': 4, 'action': None, 'reward': 1.226549956293993, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.23)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 1.07286346394
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 30, 't': 5, 'action': None, 'reward': 1.0728634639433126, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.07)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 1.8541426025
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 29, 't': 6, 'action': 'forward', 'reward': 1.854142602499749, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.85)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 1.88051178468
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 28, 't': 7, 'action': None, 'reward': 1.8805117846793478, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.88)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 1.59113663836
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 27, 't': 8, 'action': None, 'reward': 1.5911366383563865, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.59)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 1.10741568333
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 26, 't': 9, 'action': None, 'reward': 1.1074156833339877, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.11)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 2.05610889488
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 25, 't': 10, 'action': None, 'reward': 2.0561088948822945, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.06)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: 2.46668074168
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 24, 't': 11, 'action': 'forward', 'reward': 2.466680741677131, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent followed the waypoint forward. (rewarded 2.47)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: 2.61031712406
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 23, 't': 12, 'action': 'forward', 'reward': 2.610317124059489, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 2.61)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: right, reward: 0.504498137038
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 22, 't': 13, 'action': 'right', 'reward': 0.5044981370375162, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.50)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: 1.91408169832
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 21, 't': 14, 'action': 'forward', 'reward': 1.9140816983171405, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.91)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: forward, reward: -9.86709132973
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 20, 't': 15, 'action': 'forward', 'reward': -9.867091329727385, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving forward through a red light. (rewarded -9.87)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: None, reward: 2.29090395088
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 19, 't': 16, 'action': None, 'reward': 2.290903950877021, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.29)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: forward, reward: 0.978700497665
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 18, 't': 17, 'action': 'forward', 'reward': 0.9787004976653486, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.98)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: 0.853803175633
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'right'), 'deadline': 17, 't': 18, 'action': 'right', 'reward': 0.8538031756333682, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'right')
Agent drove right instead of forward. (rewarded 0.85)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 0.534860577968
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 16, 't': 19, 'action': 'right', 'reward': 0.5348605779680023, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 0.53)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: right, reward: 1.60740599974
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 15, 't': 20, 'action': 'right', 'reward': 1.6074059997375882, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.61)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: -19.7103633672
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 14, 't': 21, 'action': 'left', 'reward': -19.710363367204263, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.71)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: 0.493506873377
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 13, 't': 22, 'action': 'forward', 'reward': 0.4935068733768857, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent drove forward instead of right. (rewarded 0.49)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.5860931584
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 23, 'action': None, 'reward': 1.5860931583988154, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.59)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.33828105888
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 24, 'action': None, 'reward': 1.3382810588752796, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.34)
29% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 2.35113996921
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 25, 'action': 'right', 'reward': 2.351139969212227, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.35)
26% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 0.632880178474
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 9, 't': 26, 'action': None, 'reward': 0.6328801784736646, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 0.63)
23% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 0.928361218118
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 8, 't': 27, 'action': None, 'reward': 0.9283612181180865, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.93)
20% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 0.94856006254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'left'), 'deadline': 7, 't': 28, 'action': 'right', 'reward': 0.9485600625401132, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 0.95)
17% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.90875075892
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 6, 't': 29, 'action': None, 'reward': 1.908750758920966, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.91)
14% of time remaining to reach destination.
/-------------------
| Step 30 Results
\-------------------
Environment.step(): t = 30
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.95797454379
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 30, 'action': None, 'reward': 1.9579745437861102, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.96)
11% of time remaining to reach destination.
/-------------------
| Step 31 Results
\-------------------
Environment.step(): t = 31
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 2.07271240782
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 4, 't': 31, 'action': None, 'reward': 2.0727124078196466, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.07)
9% of time remaining to reach destination.
/-------------------
| Step 32 Results
\-------------------
Environment.step(): t = 32
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 2.04848147971
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 3, 't': 32, 'action': None, 'reward': 2.048481479708305, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.05)
6% of time remaining to reach destination.
/-------------------
| Step 33 Results
\-------------------
Environment.step(): t = 33
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 0.919487741244
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 33, 'action': 'left', 'reward': 0.9194877412437346, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.92)
3% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 197
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (4, 4), deadline = 20
Simulating trial. . .
epsilon = 0.1395; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 2.78506882991
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', 'right'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.785068829912566, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', 'right')
Agent followed the waypoint right. (rewarded 2.79)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 1.12189302735
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.1218930273458245, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.12)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 1.11277637564
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.1127763756362221, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.11)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 1.43823699196
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.4382369919569635, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.44)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 1.95113075774
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.9511307577362824, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.95)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 1.64771482445
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.6477148244519755, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.65)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 1.48803808733
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.4880380873342938, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.49)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 1.7861547932
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.786154793203048, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.79)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 2.09410843467
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.094108434672123, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.09)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 0.800581101748
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.8005811017477502, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.80)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: None, reward: 1.55302597855
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.5530259785453995, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.55)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 0.824012611912
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.8240126119120688, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.82)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: forward, reward: 0.553428437962
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 0.5534284379618353, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent drove forward instead of right. (rewarded 0.55)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 1.41893312516
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.4189331251627113, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.42)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 1.70948323584
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.7094832358433463, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.71)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 1.72782795015
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.7278279501538838, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.73)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: forward, reward: 0.471661211497
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 0.4716612114968757, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.47)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: right, reward: 0.511522340804
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.5115223408038596, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.51)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 0.317435099008
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 0.3174350990079584, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.32)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: 1.90867398325
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 1.9086739832547037, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.91)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 198
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (1, 3), deadline = 25
Simulating trial. . .
epsilon = 0.1381; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 2.37527148203
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.3752714820275598, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.38)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.1881495968
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.188149596796567, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 1.19)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 2.86447824259
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.864478242588331, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.86)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.29249765451
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.2924976545139164, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 1.29)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: forward, reward: 2.20615526903
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.206155269025788, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.21)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: left, reward: 1.52244357436
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.5224435743595848, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove left instead of forward. (rewarded 1.52)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 1.2618082628
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.2618082628037253, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.26)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 2.62244141847
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 2.6224414184730134, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.62)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.5091535519
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.5091535519047632, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.51)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 2.37400184377
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.3740018437662793, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.37)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 2.58429485762
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.5842948576157054, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 2.58)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: forward, reward: 1.69785375168
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.697853751681346, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent followed the waypoint forward. (rewarded 1.70)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 1.97116255715
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.9711625571451672, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.97)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: forward, reward: 2.23142931401
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 2.231429314010089, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.23)
44% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 199
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (5, 2), deadline = 35
Simulating trial. . .
epsilon = 0.1367; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1367; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 2.00380697914
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 35, 't': 0, 'action': None, 'reward': 2.0038069791384494, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.00)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 1.15765461324
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 34, 't': 1, 'action': None, 'reward': 1.1576546132388577, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.16)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 2.46113447767
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 33, 't': 2, 'action': None, 'reward': 2.4611344776689617, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.46)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 2.56144960011
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'forward'), 'deadline': 32, 't': 3, 'action': 'right', 'reward': 2.56144960010576, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'forward')
Agent followed the waypoint right. (rewarded 2.56)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 1.51440408949
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 31, 't': 4, 'action': 'right', 'reward': 1.5144040894864819, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 1.51)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.84672086788
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 30, 't': 5, 'action': None, 'reward': 1.8467208678801945, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.85)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.56694054386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 29, 't': 6, 'action': None, 'reward': 1.5669405438626138, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.57)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 2.43699893392
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 28, 't': 7, 'action': 'forward', 'reward': 2.43699893391787, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.44)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 1.81056722166
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 27, 't': 8, 'action': 'right', 'reward': 1.8105672216627489, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent drove right instead of forward. (rewarded 1.81)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 2.79424265887
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 26, 't': 9, 'action': None, 'reward': 2.7942426588708966, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.79)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 2.4757973098
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 25, 't': 10, 'action': None, 'reward': 2.4757973098048938, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.48)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: None, reward: 1.20956644122
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 11, 'action': None, 'reward': 1.209566441221604, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.21)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.5295126049
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'right'), 'deadline': 23, 't': 12, 'action': 'right', 'reward': 1.5295126049017331, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'right')
Agent drove right instead of left. (rewarded 1.53)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: -40.4819384243
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'right'), 'deadline': 22, 't': 13, 'action': 'left', 'reward': -40.481938424348414, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.48)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.47068369547
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'right'), 'deadline': 21, 't': 14, 'action': None, 'reward': 2.4706836954700826, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 2.47)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 0.959247681472
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 20, 't': 15, 'action': 'right', 'reward': 0.9592476814715833, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.96)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 1.96130170697
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 16, 'action': 'right', 'reward': 1.9613017069696812, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.96)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.59354722912
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 17, 'action': None, 'reward': 1.5935472291213915, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.59)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: -10.3952358729
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 18, 'action': 'left', 'reward': -10.395235872916164, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent attempted driving left through a red light. (rewarded -10.40)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 0.38579344093
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'left'), 'deadline': 16, 't': 19, 'action': None, 'reward': 0.3857934409302831, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.39)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: -5.6850370407
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'right'), 'deadline': 15, 't': 20, 'action': None, 'reward': -5.685037040699113, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -5.69)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 0.954082900223
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 14, 't': 21, 'action': 'forward', 'reward': 0.9540829002227822, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 0.95)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: None, reward: 1.22992529821
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 22, 'action': None, 'reward': 1.2299252982094968, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.23)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: forward, reward: 1.27075560382
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 12, 't': 23, 'action': 'forward', 'reward': 1.270755603815981, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 1.27)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 0.861273761036
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 11, 't': 24, 'action': None, 'reward': 0.8612737610362748, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.86)
29% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: left, reward: 2.44497274099
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 10, 't': 25, 'action': 'left', 'reward': 2.4449727409925606, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 2.44)
26% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: 2.15756483131
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 9, 't': 26, 'action': None, 'reward': 2.157564831308453, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.16)
23% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: None, reward: -5.20275046606
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 8, 't': 27, 'action': None, 'reward': -5.20275046605759, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.20)
20% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: forward, reward: 1.42725960661
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 7, 't': 28, 'action': 'forward', 'reward': 1.4272596066068677, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.43)
17% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 200
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (7, 6), deadline = 20
Simulating trial. . .
epsilon = 0.1353; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.2321748769
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.2321748769016903, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.23)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.81299922772
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.8129992277193008, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.81)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.87702884997
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.8770288499682293, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.88)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.07530260485
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.075302604849032, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.08)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: right, reward: 0.889578073388
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.8895780733882418, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 0.89)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: right, reward: 1.17787043662
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.1778704366226316, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 1.18)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: right, reward: 1.04247636246
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'right'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.0424763624635571, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'right')
Agent followed the waypoint right. (rewarded 1.04)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: forward, reward: 2.58398778172
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.58398778171544, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 2.58)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.84984381575
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.8498438157529755, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.85)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.68577181271
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.6857718127131198, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.69)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: None, reward: 1.74792462512
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.7479246251167695, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.75)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: left, reward: 2.12088080565
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 2.120880805647858, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 2.12)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: -5.64258209228
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', None), 'deadline': 8, 't': 12, 'action': None, 'reward': -5.642582092277782, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.64)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 2.22863980782
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 2.228639807820404, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.23)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 0.697329839189
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.6973298391891378, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.70)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: -5.00205840902
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': -5.002058409018659, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.00)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: forward, reward: 1.24284954026
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.2428495402624349, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.24)
15% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 201
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (8, 2), deadline = 25
Simulating trial. . .
epsilon = 0.1340; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: right, reward: 2.03183895618
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.0318389561810664, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.03)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.2234544849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.2234544848983266, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.22)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.83866790661
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.8386679066107936, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.84)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.08016397624
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.080163976239336, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.08)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 2.49964637384
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.4996463738357737, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.50)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.13735463885
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.1373546388501796, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.14)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 1.51100361041
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.5110036104148241, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 1.51)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: None, reward: 2.22906326415
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.2290632641485777, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.23)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: right, reward: 0.583483652091
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.5834836520907917, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent drove right instead of forward. (rewarded 0.58)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 2.13811860953
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.1381186095341382, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.14)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 2.58596689031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.5859668903108557, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.59)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: left, reward: 1.21659132425
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.2165913242549613, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.22)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 2.66592091785
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.66592091785087, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.67)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: forward, reward: 0.2888372165
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 0.2888372165000572, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent drove forward instead of left. (rewarded 0.29)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: 2.40449867256
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 14, 'action': 'left', 'reward': 2.404498672561682, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.40)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 202
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (2, 6), deadline = 20
Simulating trial. . .
epsilon = 0.1327; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 1.88025583048
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.8802558304798858, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.88)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 1.32151657468
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.3215165746848592, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.32)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 2.49370420758
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.493704207576495, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.49)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 2.52427361213
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.5242736121346816, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.52)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 1.68735531815
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.6873553181476213, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.69)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: left, reward: 1.76146253577
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.761462535765649, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.76)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: -4.43341552189
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 14, 't': 6, 'action': None, 'reward': -4.433415521893302, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.43)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 0.639347488089
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.6393474880887366, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.64)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: right, reward: 1.54015654972
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.5401565497212308, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 1.54)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: right, reward: 1.33861190468
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.338611904676561, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.34)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 2.38162724526
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.3816272452567246, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.38)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 2.05376749783
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.053767497832144, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.05)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 2.45528778302
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.455287783023234, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.46)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 2.26741622137
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 2.267416221372934, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 2.27)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 2.10754202219
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 2.107542022190314, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.11)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: -0.129571679867
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': -0.12957167986698814, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent drove right instead of left. (rewarded -0.13)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 2.31216720291
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 2.312167202908282, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 2.31)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 0.798200759926
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': 0.7982007599263649, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 0.80)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: left, reward: 0.20205725854
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': 0.20205725854034706, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent drove left instead of right. (rewarded 0.20)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.75630653918
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.7563065391843617, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.76)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 203
\-------------------------
Environment.reset(): Trial set up with start = (4, 7), destination = (8, 6), deadline = 25
Simulating trial. . .
epsilon = 0.1313; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: right, reward: 1.87591852863
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.8759185286291755, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove right instead of left. (rewarded 1.88)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: left, reward: 1.01465507642
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 1.0146550764241664, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent drove left instead of forward. (rewarded 1.01)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 1.83422916244
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.8342291624407623, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.83)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 1.18188920086
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.1818892008570352, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.18)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 1.70356865105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.7035686510511026, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.70)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 1.12803990514
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.1280399051403263, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.13)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 0.114330764418
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.11433076441824352, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.11)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: left, reward: 1.44027671325
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 1.4402767132542584, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.44)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.50555743594
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.5055574359366586, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.51)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: forward, reward: 2.16342247266
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.1634224726582643, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.16)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: right, reward: 1.00872157402
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.008721574023602, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent drove right instead of left. (rewarded 1.01)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 2.4187563508
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'forward'), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.4187563508001806, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.42)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 2.029657699
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 2.029657699001704, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.03)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 1.31484628029
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'forward'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.3148462802918528, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'forward')
Agent followed the waypoint right. (rewarded 1.31)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 2.13698141254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 11, 't': 14, 'action': None, 'reward': 2.1369814125438795, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.14)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: right, reward: 2.05984007215
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'forward'), 'deadline': 10, 't': 15, 'action': 'right', 'reward': 2.0598400721512613, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 2.06)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: 1.4677244213
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 1.4677244213037426, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.47)
32% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 204
\-------------------------
Environment.reset(): Trial set up with start = (2, 7), destination = (6, 6), deadline = 25
Simulating trial. . .
epsilon = 0.1300; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 2.92284784839
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.922847848394776, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.92)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.94395540064
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.9439554006357187, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.94)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 1.09423935412
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.0942393541204634, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.09)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: forward, reward: -10.1537338578
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': -10.153733857811757, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.15)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: None, reward: 2.51649362088
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.5164936208821365, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 2.52)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 2.80627640519
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.8062764051943554, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.81)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 2.464062062
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.4640620619958775, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.46)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 2.21641758275
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.2164175827457724, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.22)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 2.36813097915
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'right'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.36813097915295, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'right')
Agent properly idled at a red light. (rewarded 2.37)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 2.04869318246
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.048693182459031, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.05)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.67489370383
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.6748937038266523, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.67)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 1.27262637313
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.2726263731340304, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.27)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 1.3140049698
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.3140049698028455, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.31)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: forward, reward: 2.41094961585
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 2.410949615848538, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.41)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 1.33691371359
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 1.336913713587235, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.34)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 205
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (2, 7), deadline = 30
Simulating trial. . .
epsilon = 0.1287; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 2.20307658549
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.2030765854868335, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.20)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 1.22113675648
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.2211367564839941, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.22)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 2.5926195578
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.5926195577955866, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.59)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 1.87611116183
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 1.8761111618337158, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.88)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: 2.88816398699
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 2.8881639869933022, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.89)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 0.134911953314
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 0.1349119533143598, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent drove right instead of forward. (rewarded 0.13)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 1.79911504568
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.799115045676266, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.80)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: forward, reward: 0.0142258163442
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 0.014225816344171194, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent drove forward instead of left. (rewarded 0.01)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: right, reward: 1.05092769571
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 1.050927695712598, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent drove right instead of left. (rewarded 1.05)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 1.5977481864
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 1.5977481864025762, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.60)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 2.54217512715
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 2.542175127151829, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 2.54)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: 1.87168679634
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 1.8716867963415311, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.87)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 1.19625637678
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 18, 't': 12, 'action': 'forward', 'reward': 1.1962563767811654, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.20)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.56085261107
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 13, 'action': None, 'reward': 2.5608526110710192, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.56)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.4050020106
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.4050020106024763, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.41)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.47023507302
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 15, 'action': None, 'reward': 2.470235073024864, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.47)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 0.799977939083
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 14, 't': 16, 'action': None, 'reward': 0.7999779390825097, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.80)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: left, reward: 0.788813003096
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'left'), 'deadline': 13, 't': 17, 'action': 'left', 'reward': 0.7888130030956051, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'left')
Agent drove left instead of forward. (rewarded 0.79)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 2.30230867531
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'forward'), 'deadline': 12, 't': 18, 'action': 'right', 'reward': 2.3023086753134328, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.30)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 0.774022002402
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 0.774022002401598, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.77)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: -5.53916518445
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 10, 't': 20, 'action': None, 'reward': -5.5391651844475165, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.54)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 1.57458664069
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 21, 'action': None, 'reward': 1.57458664068552, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.57)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 1.34080745112
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 8, 't': 22, 'action': None, 'reward': 1.3408074511218286, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.34)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 0.545993898175
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 0.5459938981746358, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove right instead of forward. (rewarded 0.55)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 0.449726892653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 0.4497268926529707, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 0.45)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.50043501424
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 5, 't': 25, 'action': None, 'reward': 1.5004350142388587, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.50)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.01724796969
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 4, 't': 26, 'action': None, 'reward': 1.0172479696920504, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.02)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 0.952891318834
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 3, 't': 27, 'action': None, 'reward': 0.9528913188340353, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.95)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 0.386700234064
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 2, 't': 28, 'action': 'right', 'reward': 0.3867002340640433, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.39)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 0.262008794416
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 1, 't': 29, 'action': 'right', 'reward': 0.26200879441614267, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.26)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 206
\-------------------------
Environment.reset(): Trial set up with start = (4, 7), destination = (2, 3), deadline = 20
Simulating trial. . .
epsilon = 0.1275; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1275; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1275; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.54437253045
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.5443725304527396, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.54)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.94570853883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.9457085388337094, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.95)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 1.76271177297
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.7627117729654702, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.76)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 1.43495014296
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.4349501429637006, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.43)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 1.75031237705
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.7503123770519313, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.75)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: left, reward: 0.114482490163
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 0.11448249016269396, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent drove left instead of forward. (rewarded 0.11)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 1.88532120238
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.8853212023814585, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.89)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: 2.59092788235
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 2.5909278823456083, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.59)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 1.67749377871
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.677493778705047, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.68)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 2.30564845308
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.30564845308261, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.31)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 1.94929357612
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.9492935761172834, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.95)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 1.20832343701
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.2083234370096374, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.21)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 1.60145670386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.6014567038572358, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.60)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: left, reward: 1.00665620605
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.0066562060489892, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.01)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 207
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (3, 2), deadline = 25
Simulating trial. . .
epsilon = 0.1262; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: left, reward: 1.95905373806
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 1.9590537380635993, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.96)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: forward, reward: 2.4721480953
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 2.472148095302218, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 2.47)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: forward, reward: 2.18157499878
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 2.1815749987761235, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 2.18)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: right, reward: 1.50420254488
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.504202544884023, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.50)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: None, reward: 1.70696766542
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.706967665421618, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.71)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: None, reward: 1.73866020972
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.738660209717984, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.74)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: forward, reward: 1.37247780781
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.3724778078142217, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.37)
72% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 208
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (3, 7), deadline = 25
Simulating trial. . .
epsilon = 0.1249; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1249; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1249; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1249; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 0.222313875808
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'right'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 0.2223138758083434, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'right')
Agent drove forward instead of right. (rewarded 0.22)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: left, reward: 0.924086093903
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 24, 't': 1, 'action': 'left', 'reward': 0.9240860939032722, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove left instead of right. (rewarded 0.92)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: -5.44540621664
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 23, 't': 2, 'action': None, 'reward': -5.4454062166418655, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.45)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.95941729915
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.9594172991497736, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.96)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.51937156797
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.51937156796616, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.52)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 1.08153971562
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'right'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.081539715619246, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'right')
Agent drove right instead of left. (rewarded 1.08)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.41274697379
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.4127469737921974, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.41)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.65306749422
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.6530674942195263, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.65)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.59202398821
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.5920239882135947, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.59)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: left, reward: 0.909193651745
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 0.9091936517450385, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.91)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: 1.03832365702
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 1.0383236570225642, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.04)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 1.23327089649
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.233270896493075, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.23)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 0.80956962689
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 0.8095696268904375, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 0.81)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: right, reward: 1.0688471064
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.0688471064041065, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove right instead of forward. (rewarded 1.07)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 1.10285331874
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.1028533187376965, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.10)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: -40.6044502004
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'left'), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': -40.60445020038734, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.60)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: None, reward: 0.27362457228
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', 'left'), 'deadline': 9, 't': 16, 'action': None, 'reward': 0.2736245722795848, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.27)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: 2.20223353905
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'right', None), 'deadline': 8, 't': 17, 'action': 'left', 'reward': 2.202233539050641, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'right', None)
Agent followed the waypoint left. (rewarded 2.20)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 2.46480598305
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': 2.4648059830513382, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 2.46)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: left, reward: -39.9815347149
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'forward'), 'deadline': 6, 't': 19, 'action': 'left', 'reward': -39.98153471487573, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.98)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 0.548789170316
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 0.5487891703159373, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 0.55)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: 1.35066399949
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 4, 't': 21, 'action': 'forward', 'reward': 1.35066399949232, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent followed the waypoint forward. (rewarded 1.35)
12% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 209
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (8, 7), deadline = 25
Simulating trial. . .
epsilon = 0.1237; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1237; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1237; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1237; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: right, reward: 2.3463212841
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'right'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.346321284102056, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'right')
Agent followed the waypoint right. (rewarded 2.35)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 0.695557468868
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.6955574688681583, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 0.70)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: right, reward: 0.3562959425
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 0.3562959424995076, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 0.36)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: left, reward: 1.37389305941
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 1.3738930594137881, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.37)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: None, reward: 2.78855221968
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.788552219676933, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.79)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: left, reward: 2.2109984761
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 2.210998476102854, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.21)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: left, reward: 1.91530811107
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.9153081110669135, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded 1.92)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: forward, reward: -40.9176301749
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': -40.91763017494646, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.92)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.66212769495
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.6621276949510173, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.66)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.96844285693
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.9684428569254533, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.97)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.22961165878
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.2296116587819776, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.23)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 1.14347297144
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.1434729714402572, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.14)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 2.53787916584
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.5378791658394135, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.54)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: 1.20561493671
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 1.2056149367072864, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.21)
44% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 210
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (8, 7), deadline = 20
Simulating trial. . .
epsilon = 0.1225; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.2284704182
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.2284704182042716, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.23)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: 2.51003629606
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 2.510036296060618, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.51)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: left, reward: 1.14724891144
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.1472489114364754, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.15)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 1.74204209863
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.7420420986288065, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.74)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 2.01148450359
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.0114845035934046, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.01)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: 1.01132854742
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.0113285474210219, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.01)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.47519407199
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.475194071991671, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.48)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 0.938392833705
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 13, 't': 7, 'action': None, 'reward': 0.9383928337047331, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.94)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 2.56506199994
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.565061999938435, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.57)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 0.959552824173
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.9595528241726388, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 0.96)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 211
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (6, 5), deadline = 20
Simulating trial. . .
epsilon = 0.1212; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: forward, reward: 1.27760560621
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.277605606207878, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 1.28)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: 1.8079158744
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.807915874404923, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent drove forward instead of left. (rewarded 1.81)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: left, reward: 1.21280475092
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.2128047509160291, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.21)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: left, reward: -0.0325790907079
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': -0.0325790907079323, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent drove left instead of forward. (rewarded -0.03)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 1.60893802415
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.608938024154768, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.61)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 0.983594388613
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.9835943886125118, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 0.98)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.22060249257
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.220602492574276, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.22)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 1.16410821027
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.1641082102698737, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent drove right instead of forward. (rewarded 1.16)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 2.05927136081
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.0592713608117195, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.06)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: left, reward: 2.18926452491
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 2.1892645249106133, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.19)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: right, reward: -0.0253779708819
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'right'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': -0.0253779708819023, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'right')
Agent drove right instead of left. (rewarded -0.03)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: 1.55335745658
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.5533574565793251, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.55)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: None, reward: 2.12145263505
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.1214526350491147, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.12)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: right, reward: 1.88086989854
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.8808698985444565, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 1.88)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 0.949356431033
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.9493564310327052, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.95)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: -9.2242167353
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 5, 't': 15, 'action': 'left', 'reward': -9.224216735302685, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent attempted driving left through a red light. (rewarded -9.22)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: right, reward: 1.30204960374
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.302049603737141, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.30)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: right, reward: 1.70719845654
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'left'), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 1.7071984565440874, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'left')
Agent followed the waypoint right. (rewarded 1.71)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: left, reward: 0.857773705397
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': 0.8577737053969514, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.86)
5% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 212
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (2, 7), deadline = 30
Simulating trial. . .
epsilon = 0.1200; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 2.79952863215
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 30, 't': 0, 'action': None, 'reward': 2.7995286321473727, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.80)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 1.00617667367
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'forward', None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.0061766736749005, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'forward', None)
Agent followed the waypoint right. (rewarded 1.01)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: None, reward: 1.92099024867
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.920990248665652, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.92)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: right, reward: 2.55093133868
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 2.5509313386752748, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 2.55)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: right, reward: 0.0412859601502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': 'right', 'reward': 0.041285960150165946, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.04)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 2.08800801794
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.0880080179373306, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.09)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 2.89150979182
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.8915097918232626, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.89)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: -5.93436900456
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 23, 't': 7, 'action': None, 'reward': -5.934369004562209, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.93)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: left, reward: 1.73828076943
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': 1.738280769434364, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.74)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: right, reward: 1.84359648226
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 1.8435964822599589, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove right instead of forward. (rewarded 1.84)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: right, reward: 1.38192096677
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.38192096677009, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.38)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: None, reward: 1.03238272021
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.0323827202066442, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.03)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 2.79682719694
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'right'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 2.7968271969391596, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'right')
Agent followed the waypoint right. (rewarded 2.80)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: right, reward: 1.30186858222
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'left'), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 1.301868582223597, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.30)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 2.20406344189
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 16, 't': 14, 'action': 'forward', 'reward': 2.2040634418943528, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 2.20)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.86627161345
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 15, 't': 15, 'action': None, 'reward': 1.8662716134546664, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.87)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 2.21690295325
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 16, 'action': 'forward', 'reward': 2.2169029532480815, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.22)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: -4.60371952739
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 17, 'action': None, 'reward': -4.6037195273927285, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.60)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: left, reward: 0.837443756105
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 12, 't': 18, 'action': 'left', 'reward': 0.8374437561054662, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.84)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: right, reward: 0.303866499452
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 0.30386649945220345, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.30)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: 0.611467568951
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 10, 't': 20, 'action': 'left', 'reward': 0.6114675689506197, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 0.61)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: None, reward: 2.20322716604
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 9, 't': 21, 'action': None, 'reward': 2.203227166042483, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.20)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: 0.597974038544
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 8, 't': 22, 'action': 'left', 'reward': 0.5979740385435011, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.60)
23% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 213
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (6, 5), deadline = 30
Simulating trial. . .
epsilon = 0.1188; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 2.17247324973
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 2.172473249725943, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.17)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 1.1059223885
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.1059223885033314, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.11)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 1.37547313321
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.375473133214488, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.38)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 1.38687998875
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 1.3868799887450567, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.39)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: left, reward: 1.58663333529
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 1.5866333352887385, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent drove left instead of forward. (rewarded 1.59)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: right, reward: 1.19618226177
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.1961822617720073, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.20)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.56834298004
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.5683429800400894, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.57)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 1.75129566569
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 23, 't': 7, 'action': 'forward', 'reward': 1.751295665685284, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.75)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: 1.35397995269
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': 1.353979952694316, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.35)
70% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 214
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (8, 6), deadline = 20
Simulating trial. . .
epsilon = 0.1177; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: 2.37267592591
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.3726759259086316, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.37)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: 2.38864634656
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 2.388646346559576, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.39)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.18900622594
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.1890062259378062, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.19)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.85475437628
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.8547543762761167, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.85)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: right, reward: 0.0272256063088
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.02722560630876425, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 0.03)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: right, reward: 2.085508032
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.0855080319982386, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.09)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 2.37039925176
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.3703992517647317, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 2.37)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 0.887717924519
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'right', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.8877179245187203, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'right', None)
Agent followed the waypoint right. (rewarded 0.89)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: left, reward: 2.80504339203
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 2.8050433920263993, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.81)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.01358223689
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.013582236891153, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.01)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.39612922306
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.3961292230639133, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.40)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: forward, reward: 1.0070386871
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.0070386871039867, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.01)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 215
\-------------------------
Environment.reset(): Trial set up with start = (2, 6), destination = (6, 4), deadline = 30
Simulating trial. . .
epsilon = 0.1165; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 1.60571520938
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.6057152093817983, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.61)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: 1.97001210782
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 1.9700121078187882, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.97)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 2.37617876292
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 2.3761787629167035, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.38)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 1.26753005738
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 1.2675300573752946, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.27)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: -10.4756259034
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': -10.475625903405533, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.48)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 2.78535126724
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 2.7853512672352645, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.79)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: None, reward: 1.50424372044
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.5042437204353267, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.50)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: -0.0298629904313
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'right'), 'deadline': 23, 't': 7, 'action': 'right', 'reward': -0.029862990431335334, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'right')
Agent drove right instead of forward. (rewarded -0.03)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: left, reward: 2.13327726074
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': 2.1332772607419246, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.13)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: left, reward: -40.6403599308
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': -40.64035993082303, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'right', None)
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.64)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 2.7837530461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 20, 't': 10, 'action': None, 'reward': 2.783753046103456, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 2.78)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 1.80879584489
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 19, 't': 11, 'action': 'right', 'reward': 1.8087958448922734, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.81)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.87637624843
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.8763762484250477, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.88)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 0.829580756572
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'right', 'reward': 0.8295807565716333, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.83)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 1.52807312391
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 16, 't': 14, 'action': 'right', 'reward': 1.5280731239124596, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.53)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: 1.56543019543
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': 1.5654301954257752, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.57)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 1.42407007887
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 14, 't': 16, 'action': None, 'reward': 1.4240700788701866, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.42)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: right, reward: 2.31471346039
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 17, 'action': 'right', 'reward': 2.3147134603949517, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.31)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 216
\-------------------------
Environment.reset(): Trial set up with start = (7, 4), destination = (1, 2), deadline = 20
Simulating trial. . .
epsilon = 0.1153; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1153; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 2.65718901345
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.657189013454354, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 2.66)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: right, reward: 1.10001371443
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.1000137144344788, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.10)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 1.20693727574
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.2069372757408299, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.21)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: 2.61862962127
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 2.6186296212655895, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 2.62)
80% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 217
\-------------------------
Environment.reset(): Trial set up with start = (5, 5), destination = (1, 6), deadline = 25
Simulating trial. . .
epsilon = 0.1142; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1142; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 1.72333405253
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'right'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.7233340525336804, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 1.72)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 1.50160156035
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.5016015603510442, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.50)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 1.01461267709
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.0146126770869475, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.01)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 1.38295720409
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.3829572040921372, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.38)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: forward, reward: -9.47513433538
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -9.475134335378865, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent attempted driving forward through a red light. (rewarded -9.48)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: None, reward: 2.86612963243
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.866129632428639, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.87)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: right, reward: 1.02304641746
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.023046417455284, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.02)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: 2.27350205918
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 2.2735020591752777, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 2.27)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 1.17725279875
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.1772527987519634, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.18)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: 1.59913612629
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.5991361262945234, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent followed the waypoint forward. (rewarded 1.60)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: -10.0909975612
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 15, 't': 10, 'action': 'left', 'reward': -10.09099756117078, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent attempted driving left through a red light. (rewarded -10.09)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 1.43063231364
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.4306323136422865, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.43)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: left, reward: 1.0249639462
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 13, 't': 12, 'action': 'left', 'reward': 1.0249639461958635, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.02)
48% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 218
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (3, 3), deadline = 20
Simulating trial. . .
epsilon = 0.1130; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 1.33723164874
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.3372316487354112, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.34)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: right, reward: 2.28780090807
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'right'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.2878009080725716, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'right')
Agent followed the waypoint right. (rewarded 2.29)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: right, reward: -0.000445468223845
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': -0.00044546822384472495, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded -0.00)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: forward, reward: -40.8128700051
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': -40.81287000511254, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', None)
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.81)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.10943896744
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.1094389674427092, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.11)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: left, reward: 2.6348446306
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.634844630603613, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.63)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 0.428018371614
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 0.4280183716142092, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent drove forward instead of left. (rewarded 0.43)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 0.118395173644
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.11839517364437624, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove right instead of left. (rewarded 0.12)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: right, reward: 0.921660424285
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 0.9216604242852364, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.92)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: left, reward: 2.18975676084
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 2.189756760842015, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 2.19)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: 0.96310094607
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 0.9631009460700637, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 0.96)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: 1.5476070489
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.5476070489018177, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 1.55)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 219
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (7, 6), deadline = 25
Simulating trial. . .
epsilon = 0.1119; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: left, reward: 1.52092314513
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', 'left'), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 1.5209231451329686, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', 'left')
Agent drove left instead of forward. (rewarded 1.52)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: 1.43754098319
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'left'), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 1.4375409831872685, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'left')
Agent drove forward instead of right. (rewarded 1.44)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: left, reward: 0.828730556794
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'left'), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 0.8287305567938859, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'left')
Agent drove left instead of right. (rewarded 0.83)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: 0.134531738606
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'right'), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 0.13453173860592482, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'right')
Agent drove right instead of forward. (rewarded 0.13)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: forward, reward: 1.07810351962
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'right'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 1.078103519623761, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'right')
Agent drove forward instead of left. (rewarded 1.08)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: None, reward: 0.949855235069
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', 'forward'), 'deadline': 20, 't': 5, 'action': None, 'reward': 0.9498552350687182, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', 'forward')
Agent properly idled at a red light. (rewarded 0.95)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (0, -1), action: forward, reward: -10.4240959851
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': -10.424095985145245, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -10.42)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 4), heading: (1, 0), action: right, reward: 1.0088989129
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.008898912899608, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.01)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: right, reward: 0.597791171851
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'right'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 0.5977911718509598, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'right')
Agent drove right instead of forward. (rewarded 0.60)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 2.03812383118
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.0381238311810588, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.04)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 2.07705570897
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.0770557089715593, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.08)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: -0.0761779146293
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': -0.07617791462930124, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded -0.08)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 1.57267657529
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.5726765752866758, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.57)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: 2.56586282299
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 2.5658628229856553, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.57)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 0.998997452622
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 0.998997452622453, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.00)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 1.82387915947
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'right'), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 1.8238791594700892, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'right')
Agent followed the waypoint forward. (rewarded 1.82)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.12377692716
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.1237769271554225, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.12)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 0.653428708219
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 0.6534287082190919, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 0.65)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: -0.11675745828
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 7, 't': 18, 'action': 'right', 'reward': -0.11675745827993711, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove right instead of left. (rewarded -0.12)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 1.34518937827
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 6, 't': 19, 'action': 'right', 'reward': 1.3451893782676192, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.35)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: left, reward: -9.69312352951
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 5, 't': 20, 'action': 'left', 'reward': -9.693123529505964, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent attempted driving left through a red light. (rewarded -9.69)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.18252084973
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 1.1825208497324928, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.18)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 2.06795850989
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 3, 't': 22, 'action': 'right', 'reward': 2.0679585098892463, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.07)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 0.528713254641
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 2, 't': 23, 'action': None, 'reward': 0.5287132546414124, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 0.53)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 0.880893166929
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.8808931669293105, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 0.88)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 220
\-------------------------
Environment.reset(): Trial set up with start = (5, 7), destination = (3, 4), deadline = 25
Simulating trial. . .
epsilon = 0.1108; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.90528793773
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.9052879377323833, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.91)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.66852470269
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.66852470269267, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.67)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.42772397562
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.427723975617353, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.43)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: right, reward: 1.06213029804
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.0621302980415597, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.06)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 1.65161439807
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.6516143980702622, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 1.65)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 1.12711637564
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.1271163756402185, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.13)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 1.11211191007
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.1121119100681667, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent drove forward instead of left. (rewarded 1.11)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 2.60999107325
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.6099910732466816, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.61)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: left, reward: -39.5872629138
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'right'), 'deadline': 17, 't': 8, 'action': 'left', 'reward': -39.58726291381296, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.59)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 0.228070730594
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'right'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.22807073059359606, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'right')
Agent drove right instead of forward. (rewarded 0.23)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (1, 0), action: left, reward: 2.55641041194
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': 'left', 'reward': 2.5564104119416036, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.56)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: forward, reward: 1.86944561574
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.8694456157426913, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.87)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: forward, reward: 1.71638060571
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 13, 't': 12, 'action': 'forward', 'reward': 1.7163806057089568, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.72)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: -4.32664655295
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': -4.326646552947712, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.33)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: 0.959156285132
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 0.9591562851317432, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 0.96)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.88904192095
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.8890419209479938, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.89)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: 2.10460762735
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'forward'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 2.104607627350182, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 2.10)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: 1.19093254722
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 1.1909325472249224, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.19)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: forward, reward: -10.5677700168
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': 'forward', 'reward': -10.56777001680308, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.57)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 1.61126719708
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 6, 't': 19, 'action': None, 'reward': 1.6112671970810075, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.61)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: None, reward: 1.38261347103
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.3826134710283162, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.38)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: left, reward: 1.31634917434
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'left'), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 1.3163491743386762, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'left')
Agent drove left instead of forward. (rewarded 1.32)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 1.76253422651
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 3, 't': 22, 'action': None, 'reward': 1.7625342265089865, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.76)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 0.552193828571
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 2, 't': 23, 'action': 'right', 'reward': 0.5521938285712789, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 0.55)
4% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 0.868247662239
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 1, 't': 24, 'action': None, 'reward': 0.8682476622392763, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.87)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 221
\-------------------------
Environment.reset(): Trial set up with start = (4, 6), destination = (8, 3), deadline = 35
Simulating trial. . .
epsilon = 0.1097; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: right, reward: -19.819205045
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 35, 't': 0, 'action': 'right', 'reward': -19.819205045017874, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent attempted driving right through traffic and cause a minor accident. (rewarded -19.82)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: None, reward: 2.40343152899
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 34, 't': 1, 'action': None, 'reward': 2.403431528989735, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.40)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: None, reward: 1.58301573739
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 33, 't': 2, 'action': None, 'reward': 1.5830157373899965, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.58)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: forward, reward: 2.9379413398
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 32, 't': 3, 'action': 'forward', 'reward': 2.937941339795752, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.94)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: forward, reward: 1.97904642184
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 31, 't': 4, 'action': 'forward', 'reward': 1.9790464218422825, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 1.98)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: 1.87145129574
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 30, 't': 5, 'action': 'forward', 'reward': 1.8714512957407143, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.87)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 0.911892332764
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 29, 't': 6, 'action': 'right', 'reward': 0.9118923327637909, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.91)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 2.8116487058
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 28, 't': 7, 'action': None, 'reward': 2.8116487057953012, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.81)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 1.66754159586
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'left'), 'deadline': 27, 't': 8, 'action': 'right', 'reward': 1.6675415958605782, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'left')
Agent drove right instead of left. (rewarded 1.67)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: left, reward: 1.33599675348
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 26, 't': 9, 'action': 'left', 'reward': 1.3359967534793291, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 1.34)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: left, reward: 1.93967633903
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 25, 't': 10, 'action': 'left', 'reward': 1.9396763390330949, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent followed the waypoint left. (rewarded 1.94)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 2.29631022949
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 24, 't': 11, 'action': 'forward', 'reward': 2.2963102294868385, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.30)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: left, reward: 1.47472867323
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 23, 't': 12, 'action': 'left', 'reward': 1.474728673230317, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent drove left instead of right. (rewarded 1.47)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 2.14823745846
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 22, 't': 13, 'action': 'right', 'reward': 2.14823745846313, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.15)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 1.48335153764
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'right'), 'deadline': 21, 't': 14, 'action': None, 'reward': 1.4833515376401645, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 1.48)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.21441796607
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 20, 't': 15, 'action': None, 'reward': 2.2144179660735714, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.21)
54% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: None, reward: 2.04485756029
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 19, 't': 16, 'action': None, 'reward': 2.0448575602917627, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.04)
51% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 2.75249150929
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 18, 't': 17, 'action': 'right', 'reward': 2.752491509289057, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.75)
49% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: None, reward: 2.33967774798
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 17, 't': 18, 'action': None, 'reward': 2.3396777479843713, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.34)
46% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 1.49122647717
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 16, 't': 19, 'action': 'right', 'reward': 1.4912264771711956, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.49)
43% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.90979327979
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 20, 'action': None, 'reward': 1.909793279786722, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.91)
40% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 0.965952880343
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 14, 't': 21, 'action': None, 'reward': 0.9659528803432564, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 0.97)
37% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.48747934526
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 13, 't': 22, 'action': 'right', 'reward': 1.4874793452648525, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.49)
34% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 2.44280997517
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 23, 'action': 'right', 'reward': 2.4428099751692858, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.44)
31% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (7, 4), heading: (0, -1), action: right, reward: 2.48943672668
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 24, 'action': 'right', 'reward': 2.489436726681922, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.49)
29% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: right, reward: 2.12131757501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 10, 't': 25, 'action': 'right', 'reward': 2.1213175750074265, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.12)
26% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.47462974347
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 9, 't': 26, 'action': None, 'reward': 1.4746297434707598, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.47)
23% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 0.953315754718
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 27, 'action': None, 'reward': 0.9533157547176754, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.95)
20% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: left, reward: 0.940064845124
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 28, 'action': 'left', 'reward': 0.940064845124261, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.94)
17% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 222
\-------------------------
Environment.reset(): Trial set up with start = (7, 3), destination = (4, 7), deadline = 25
Simulating trial. . .
epsilon = 0.1086; alpha = 0.5000
Simulating trial. . .
epsilon = 0.1086; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: forward, reward: 0.60718195016
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'left'), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 0.6071819501597817, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'left')
Agent drove forward instead of left. (rewarded 0.61)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 2.56289211899
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.5628921189869667, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.56)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 1.74154937747
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.7415493774670792, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.74)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 1.58903019971
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.5890301997083915, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.59)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 2.37866613515
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.378666135152456, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.38)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: None, reward: 1.38335003393
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.383350033934069, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.38)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: left, reward: 1.42547205919
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', 'left'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.4254720591851995, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', 'left')
Agent followed the waypoint left. (rewarded 1.43)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: 2.6841641554
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 2.6841641553993862, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.68)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: 2.78169628597
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'right'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 2.781696285972729, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'right')
Agent followed the waypoint forward. (rewarded 2.78)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 2.47805247271
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.4780524727069437, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.48)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 1.37984494929
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.3798449492911384, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.38)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 223
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (7, 2), deadline = 20
Simulating trial. . .
epsilon = 0.1075; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: forward, reward: 0.776402748682
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'right'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 0.7764027486821419, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'right')
Agent drove forward instead of right. (rewarded 0.78)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 1.76044962226
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.7604496222553097, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.76)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: 1.39698298154
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.3969829815404107, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.40)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: left, reward: 1.26937825637
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 1.2693782563709217, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.27)
80% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 224
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (3, 2), deadline = 20
Simulating trial. . .
epsilon = 0.1065; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: forward, reward: 2.89221588984
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.892215889844425, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent followed the waypoint forward. (rewarded 2.89)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: 1.07269746559
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.0726974655933934, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent drove forward instead of left. (rewarded 1.07)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: left, reward: 2.52422788379
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 2.5242278837912524, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.52)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: 2.89085947302
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 2.890859473020229, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 2.89)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 2.71130824172
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.7113082417187333, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.71)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 2.54361544876
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.543615448759359, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.54)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: 1.70093753412
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.7009375341161608, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', None)
Agent followed the waypoint forward. (rewarded 1.70)
65% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 225
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (4, 5), deadline = 25
Simulating trial. . .
epsilon = 0.1054; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 2.45814682878
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.458146828776676, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.46)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: forward, reward: -9.38304095427
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': -9.383040954266775, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.38)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.7805353471
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.7805353470971776, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.78)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.41714930179
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.417149301790662, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.42)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 1.36530959318
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.3653095931806387, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.37)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: right, reward: 1.78328237497
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.7832823749719606, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.78)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.10913792062
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.1091379206170133, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 1.11)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.06383859306
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.0638385930600793, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.06)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.01078877432
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.0107887743237103, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.01)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: -4.24318871815
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': -4.243188718151682, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.24)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: 2.05085772859
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 2.0508577285897553, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.05)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: right, reward: 1.6327480968
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.6327480967970898, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.63)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: None, reward: 0.989838139699
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 0.9898381396993956, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.99)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: forward, reward: 2.02437838223
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 2.0243783822309025, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.02)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 1.68161545953
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.6816154595310513, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.68)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: None, reward: 1.20870482104
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.208704821041184, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.21)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: forward, reward: 1.29915322908
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 1.2991532290780567, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.30)
32% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 226
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (3, 5), deadline = 20
Simulating trial. . .
epsilon = 0.1044; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: left, reward: 2.66969341719
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.6696934171931384, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.67)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: right, reward: 1.78955802072
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.789558020722228, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 1.79)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: left, reward: 2.79103840485
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 2.7910384048514887, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.79)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 2.72112247353
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.72112247352911, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.72)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 1.63757458246
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.637574582460041, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.64)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 2.66338309682
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.663383096817901, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.66)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.06013545035
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'forward'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.0601354503464058, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'forward')
Agent properly idled at a red light. (rewarded 1.06)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 0.291262895424
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 0.2912628954242398, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 0.29)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 2.68260190385
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.6826019038491156, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.68)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 1.76739333529
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.7673933352928162, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.77)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: 2.75750763983
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 2.757507639825219, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 2.76)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 1.84076082455
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.8407608245491165, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.84)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.01009053248
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.0100905324848304, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.01)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: -5.07593290493
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': -5.075932904926962, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.08)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: -0.159955209737
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': -0.15995520973678834, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded -0.16)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 1.41808684666
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 1.4180868466611927, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.42)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 0.501521975862
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 0.5015219758623113, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent followed the waypoint forward. (rewarded 0.50)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: left, reward: 1.65848328386
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 1.658483283862586, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.66)
10% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 227
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (1, 4), deadline = 20
Simulating trial. . .
epsilon = 0.1033; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: right, reward: 1.71694211806
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'right', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.71694211806062, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', None)
Agent followed the waypoint right. (rewarded 1.72)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: right, reward: 2.82630503364
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.8263050336355517, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.83)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 2.56564767891
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.5656476789077884, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.57)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: forward, reward: 2.00293340262
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 2.002933402620717, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.00)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 2.09851916684
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.098519166839695, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.10)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.55043237828
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.5504323782845486, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent drove right instead of forward. (rewarded 1.55)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: forward, reward: 0.266727085429
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'right'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 0.2667270854294137, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'right')
Agent drove forward instead of left. (rewarded 0.27)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 2.37749689128
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', 'right'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.3774968912759196, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', 'right')
Agent properly idled at a red light. (rewarded 2.38)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 0.982551277698
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 0.9825512776980816, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 0.98)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 0.984111392728
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.984111392727897, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 0.98)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: forward, reward: 0.467170445506
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 0.4671704455063287, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove forward instead of left. (rewarded 0.47)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: right, reward: 0.921761687944
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.9217616879440622, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', None)
Agent drove right instead of left. (rewarded 0.92)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 0.184949615208
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.18494961520783781, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent drove right instead of left. (rewarded 0.18)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: None, reward: 1.26695204763
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.2669520476296126, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.27)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 0.614677425898
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 0.6146774258977563, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 0.61)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 0.7532828112
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'left'), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 0.7532828112003582, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'left')
Agent followed the waypoint forward. (rewarded 0.75)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.2817778169
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 1.281777816900964, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove right instead of left. (rewarded 1.28)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 1.82723655414
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.8272365541417446, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.83)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 0.938791004222
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.938791004222453, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.94)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 0.848626044419
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 0.8486260444191583, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded 0.85)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 228
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (8, 6), deadline = 25
Simulating trial. . .
epsilon = 0.1023; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 2.45772427681
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.4577242768053784, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.46)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: right, reward: -20.9297222066
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': -20.929722206586895, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'left')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.93)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 1.81194902417
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.8119490241709761, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.81)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 2.65874293087
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.6587429308699697, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.66)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 1.22628848403
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.2262884840304102, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.23)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: None, reward: 2.46179067809
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.461790678093774, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.46)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: right, reward: 1.06832109262
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.0683210926177862, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.07)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: 1.05747375068
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.0574737506788376, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.06)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 2.29519634802
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.295196348024576, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.30)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 2.61814341185
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.618143411846564, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.62)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 2.45312506464
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 2.453125064637743, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent followed the waypoint forward. (rewarded 2.45)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 2.08210737259
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.0821073725850896, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.08)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 2.49992927582
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.499929275822444, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.50)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: left, reward: 0.853193218605
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 0.8531932186052633, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 0.85)
44% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 229
\-------------------------
Environment.reset(): Trial set up with start = (6, 3), destination = (5, 6), deadline = 20
Simulating trial. . .
epsilon = 0.1013; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: right, reward: 1.4301407124
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.4301407124019585, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.43)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: left, reward: 0.00595716379609
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 0.0059571637960934565, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent drove left instead of right. (rewarded 0.01)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 2.30365546281
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.3036554628116686, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 2.30)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: right, reward: 1.18928437794
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 1.1892843779384954, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.19)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 2.2393408566
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.239340856600325, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.24)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 1.71788202824
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.7178820282422258, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.72)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (0, 1), action: left, reward: 2.0464467412
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.0464467412012133, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 2.05)
65% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 230
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (6, 7), deadline = 20
Simulating trial. . .
epsilon = 0.1003; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: left, reward: 2.9751309069
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.9751309068965135, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.98)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: 1.12585681282
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.1258568128208732, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', None)
Agent followed the waypoint forward. (rewarded 1.13)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 2.29947955731
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.299479557306798, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.30)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 0.547366351894
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 0.5473663518936179, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove forward instead of left. (rewarded 0.55)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 1.27402778087
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.2740277808687872, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 1.27)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: 2.30876762789
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.3087676278921427, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 2.31)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (0, -1), action: left, reward: 1.69175022472
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 1.6917502247207286, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.69)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: forward, reward: 2.51394210948
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.5139421094827847, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.51)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: forward, reward: 1.66491444146
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.664914441455657, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.66)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 231
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (5, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0993; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: left, reward: 1.3158645518
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.3158645517983414, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.32)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: forward, reward: 0.554399185471
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'right'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 0.5543991854710949, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'right')
Agent drove forward instead of left. (rewarded 0.55)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 2.78682466367
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.786824663673152, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.79)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 1.83909621013
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.839096210125255, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.84)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 2.92485663062
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.9248566306203214, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.92)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 1.38626130492
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.3862613049204855, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.39)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 1.6884921657
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'right', 'right'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.6884921656955008, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'right', 'right')
Agent drove forward instead of right. (rewarded 1.69)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.35108942643
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.351089426427584, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.35)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 2.33859835026
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.3385983502617878, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.34)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: 1.99998688803
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.9999868880259297, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.00)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: -4.1410637446
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': -4.141063744601492, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.14)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: 1.7298610592
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.7298610592014534, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.73)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: 1.09988413976
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.0998841397578027, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.10)
35% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 232
\-------------------------
Environment.reset(): Trial set up with start = (5, 2), destination = (8, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0983; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0983; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0983; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0983; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0983; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0983; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0983; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0983; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: 0.864232204889
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.8642322048885407, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'left')
Agent drove right instead of forward. (rewarded 0.86)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: 2.92500250594
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 2.9250025059430085, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.93)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 2.29801752855
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.298017528553483, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.30)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 2.38782132001
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.3878213200135594, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.39)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: 1.58605493068
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.586054930682462, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.59)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: right, reward: 0.96251226566
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.9625122656600826, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove right instead of forward. (rewarded 0.96)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: left, reward: 2.52470349004
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.5247034900414014, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 2.52)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: left, reward: 2.53828267199
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 2.538282671988946, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.54)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 0.972157492569
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 0.9721574925694045, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 0.97)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 0.528653055656
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 0.5286530556564895, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 0.53)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: 2.48656443308
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 2.486564433078561, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.49)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 2.15956475459
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.159564754588841, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.16)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: None, reward: 1.97688857141
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.9768885714062692, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.98)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: -20.4288296761
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'forward'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': -20.428829676146197, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.43)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: 1.36600182335
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 1.366001823352774, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 1.37)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 1.26566181939
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.265661819388806, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.27)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 233
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (6, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0973; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: right, reward: 1.99893686959
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.9989368695944605, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', 'left')
Agent followed the waypoint right. (rewarded 2.00)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: left, reward: -20.3535153745
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'left'}, 'violation': 3, 'light': 'green', 'state': ('right', 'green', 'left', 'left', 'right'), 'deadline': 19, 't': 1, 'action': 'left', 'reward': -20.353515374533185, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.35)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: right, reward: 0.973077170362
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.9730771703617931, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 0.97)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 2.03995110967
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.039951109666181, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.04)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: left, reward: 1.58512909185
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.5851290918516132, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove left instead of forward. (rewarded 1.59)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 2.08287783254
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.0828778325359245, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.08)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 2.74601177747
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.7460117774701596, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 2.75)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: right, reward: 0.90158228579
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.9015822857903666, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 0.90)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.47496914007
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.4749691400726264, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.47)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: left, reward: 2.20500207996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 2.2050020799561554, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.21)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 234
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (4, 2), deadline = 30
Simulating trial. . .
epsilon = 0.0963; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 1.19364181301
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'right', 'right'), 'deadline': 30, 't': 0, 'action': 'forward', 'reward': 1.1936418130127644, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', 'right')
Agent drove forward instead of left. (rewarded 1.19)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 2.47435735774
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 2.4743573577417095, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.47)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: forward, reward: 1.78273693907
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 1.7827369390742982, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.78)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 6), heading: (-1, 0), action: forward, reward: 2.47459394711
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 2.474593947111682, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.47)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: left, reward: 1.22733033073
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 26, 't': 4, 'action': 'left', 'reward': 1.227330330725872, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.23)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 2.79244497761
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.7924449776063174, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.79)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.51227243456
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.5122724345606604, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.51)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.78798099101
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.787980991008062, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.79)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: None, reward: 1.38716589523
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.3871658952349954, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.39)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: forward, reward: 1.07125098184
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'right'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.0712509818426434, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'right')
Agent followed the waypoint forward. (rewarded 1.07)
67% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 235
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (6, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0954; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: left, reward: 2.30272880473
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 2.302728804727238, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.30)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: forward, reward: 1.47611401685
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 1.4761140168488704, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 1.48)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: right, reward: 1.52547109881
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.5254710988123765, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 1.53)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: forward, reward: 1.16369240729
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.1636924072911976, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.16)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: forward, reward: 1.45483792321
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 1.4548379232058508, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.45)
80% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 236
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (5, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0944; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: left, reward: 1.89165166312
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.8916516631176887, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.89)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 2.28639247122
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.286392471216219, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.29)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 1.6404382936
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.6404382936020465, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.64)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 0.971096997323
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 17, 't': 3, 'action': None, 'reward': 0.9710969973225811, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.97)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: None, reward: 1.59460038466
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.5946003846607422, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.59)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: forward, reward: 2.04587172873
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 2.0458717287318695, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.05)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 1.94137410266
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.9413741026598685, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.94)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 2.19343686359
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.193436863593306, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.19)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: None, reward: 2.27700277778
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.2770027777805284, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.28)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: forward, reward: 2.28537210216
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 2.2853721021614537, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.29)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 237
\-------------------------
Environment.reset(): Trial set up with start = (3, 4), destination = (6, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0935; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0935; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0935; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0935; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0935; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0935; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: 1.1909629384
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.1909629384047664, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.19)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: None, reward: 1.56145853477
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.5614585347667853, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.56)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 1.41358805641
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.413588056407572, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.41)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 1.28851311307
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.2885131130695153, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.29)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.09576115674
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.095761156742273, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.10)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: right, reward: 0.615196192468
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.6151961924677424, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 0.62)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: left, reward: -20.4430747488
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 3, 'light': 'green', 'state': ('left', 'green', None, 'right', 'right'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': -20.443074748830856, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'right')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.44)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 1.20846111458
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.2084611145812587, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.21)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: None, reward: 2.20446523972
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.204465239721929, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.20)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: right, reward: 1.53081034969
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.5308103496935617, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent drove right instead of left. (rewarded 1.53)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 2.35727792629
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.357277926288985, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.36)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 1.80428207432
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.804282074323428, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent drove right instead of left. (rewarded 1.80)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 1.27556157671
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'forward'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.2755615767095483, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 1.28)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 2.45198414194
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, 'left'), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 2.451984141942176, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, 'left')
Agent followed the waypoint right. (rewarded 2.45)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.39162079859
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'right'), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.3916207985925508, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'right')
Agent properly idled at a red light. (rewarded 1.39)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.4368918644
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.4368918643996285, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.44)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 2.42317631725
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 9, 't': 16, 'action': 'forward', 'reward': 2.423176317252377, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.42)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 1.85398021564
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 1.8539802156370626, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.85)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: right, reward: 1.52647021083
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'forward'), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 1.5264702108322314, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'forward')
Agent drove right instead of forward. (rewarded 1.53)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 0.835355908104
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 6, 't': 19, 'action': None, 'reward': 0.8353559081035338, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.84)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 2.39987303345
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 20, 'action': None, 'reward': 2.399873033454117, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.40)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (7, 3), heading: (0, -1), action: None, reward: 0.699364115553
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 4, 't': 21, 'action': None, 'reward': 0.6993641155527026, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.70)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: left, reward: 0.466544793799
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 3, 't': 22, 'action': 'left', 'reward': 0.4665447937991607, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent followed the waypoint left. (rewarded 0.47)
8% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 1.18495488871
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'right', None), 'deadline': 2, 't': 23, 'action': 'right', 'reward': 1.1849548887110213, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', None)
Agent followed the waypoint right. (rewarded 1.18)
4% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 238
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (8, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0926; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 2.07186293285
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.0718629328511016, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 2.07)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 2.01081793629
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.0108179362939236, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.01)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 1.85617053829
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.8561705382866445, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.86)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 1.80491817218
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.8049181721761773, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.80)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 1.70495331712
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.704953317117846, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.70)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: None, reward: 2.08318433325
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.0831843332458257, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.08)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 5), heading: (1, 0), action: right, reward: 1.84151889259
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.8415188925940602, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove right instead of left. (rewarded 1.84)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: forward, reward: 1.14853317263
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.1485331726318804, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.15)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: forward, reward: 1.83483582756
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 1.8348358275557826, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.83)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: None, reward: -4.85883696484
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': -4.858836964840828, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.86)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: forward, reward: 0.863764462164
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'right', 'right'), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 0.8637644621643465, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'right', 'right')
Agent followed the waypoint forward. (rewarded 0.86)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 1.46559400983
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 1.4655940098264324, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.47)
52% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 239
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (2, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0916; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 0.560320355443
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.5603203554426097, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 0.56)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 2.54875963714
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.5487596371356966, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.55)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: None, reward: 2.92648963827
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.9264896382745764, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.93)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: 1.31915641913
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 1.3191564191291014, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.32)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 1.58679401778
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.5867940177776925, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.59)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 1.7717671037
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.7717671036977363, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.77)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 2.45697432205
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.4569743220527385, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.46)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 1.54375367924
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.5437536792423838, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 1.54)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: None, reward: 2.35486328041
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.354863280414614, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.35)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: left, reward: 2.23637512958
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 2.2363751295849417, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 2.24)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 2.48670617643
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 2.486706176434371, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.49)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 0.82338736898
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'right', 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 0.8233873689802695, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.82)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: forward, reward: 1.18021233279
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': 1.1802123327893062, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.18)
35% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 240
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (4, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0907; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0907; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0907; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: -5.23521670468
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('right', 'green', None, 'right', 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': -5.235216704681934, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.24)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 1.55851230255
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.5585123025512757, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 1.56)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: forward, reward: 2.33889002873
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 2.3388900287263343, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.34)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: right, reward: 0.0145842178159
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 0.014584217815898248, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 0.01)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: 0.910429748118
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.9104297481176121, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent drove right instead of left. (rewarded 0.91)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 2.1050761765
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.1050761765004355, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.11)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 0.923259894155
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.9232598941546548, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.92)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 2.09984127912
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 2.099841279119458, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 2.10)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.55052100892
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.5505210089160002, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.55)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: None, reward: 1.77450502835
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.774505028349402, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.77)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: 2.73519721827
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'right'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 2.7351972182722086, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'right')
Agent followed the waypoint forward. (rewarded 2.74)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: left, reward: 1.79141520779
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 1.7914152077875436, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent followed the waypoint left. (rewarded 1.79)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 241
\-------------------------
Environment.reset(): Trial set up with start = (7, 7), destination = (1, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0898; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0898; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 1.58221719756
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.582217197559601, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'forward')
Agent drove right instead of left. (rewarded 1.58)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 2.84979886022
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.849798860224962, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 2.85)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 2.19978122175
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 2.1997812217490225, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.20)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 0.429365722889
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 0.42936572288933283, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.43)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 0.812053075854
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 0.8120530758535055, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.81)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.71279971461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.7127997146133547, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.71)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.80825356038
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.8082535603769525, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.81)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.55835868236
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.558358682361072, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.56)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.50581601475
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.5058160147532, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.51)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 0.646020912122
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 0.6460209121217273, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent drove forward instead of left. (rewarded 0.65)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: None, reward: 1.09986991412
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.0998699141193389, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.10)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: left, reward: 2.37367755224
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 2.373677552241641, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.37)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 5), heading: (0, -1), action: None, reward: 1.74867595712
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.7486759571151371, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.75)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: left, reward: 0.75962563266
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'left', 'reward': 0.7596256326604369, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.76)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 0.813486953859
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 11, 't': 14, 'action': None, 'reward': 0.8134869538593468, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.81)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 1.72105388028
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.7210538802817, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.72)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: right, reward: 1.41846900967
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 1.4184690096692094, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.42)
32% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 242
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (5, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0889; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: right, reward: 1.44206314628
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.4420631462757667, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.44)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.37452893376
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.3745289337621123, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.37)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: right, reward: 0.97333755324
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.9733375532402876, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'left')
Agent drove right instead of left. (rewarded 0.97)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: None, reward: 1.26960396526
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.2696039652565227, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.27)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: None, reward: 2.85721401653
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.8572140165332067, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.86)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: None, reward: 1.45303569389
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.4530356938931002, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.45)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: right, reward: 0.121205101307
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 0.12120510130660067, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.12)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: right, reward: 0.0444709996829
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.044470999682892254, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.04)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: right, reward: 2.65917305081
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.659173050812951, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.66)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: right, reward: -20.2043688466
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 3, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': -20.204368846644908, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'forward')
Agent attempted driving right through traffic and cause a minor accident. (rewarded -20.20)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: right, reward: 0.307239008754
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 0.3072390087538429, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 0.31)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: None, reward: 0.888725064167
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'forward'), 'deadline': 9, 't': 11, 'action': None, 'reward': 0.8887250641672826, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'forward')
Agent properly idled at a red light. (rewarded 0.89)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: forward, reward: -39.4301681987
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', 'forward'), 'deadline': 8, 't': 12, 'action': 'forward', 'reward': -39.43016819868874, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', 'forward')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.43)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: right, reward: -0.0627392030626
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': -0.06273920306260805, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded -0.06)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 5), heading: (1, 0), action: None, reward: 1.6834076412
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.6834076412033414, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.68)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: right, reward: -0.169415265874
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': -0.1694152658743402, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent drove right instead of left. (rewarded -0.17)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: forward, reward: -9.94749155212
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', 'left', None), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': -9.94749155212098, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'left', None)
Agent attempted driving forward through a red light. (rewarded -9.95)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: right, reward: 0.614614826195
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.6146148261946243, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.61)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 2.01486290121
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 2.0148629012054924, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.01)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: None, reward: 1.91470901919
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 1.914709019193582, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.91)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 243
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (4, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0880; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: left, reward: -40.2086701963
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -40.20867019633518, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.21)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 1.5755290578
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.575529057797656, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.58)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: left, reward: 1.97639747985
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.9763974798516364, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.98)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: left, reward: 2.61236754047
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 2.612367540471852, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.61)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: forward, reward: 1.46564121173
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.4656412117327886, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.47)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 1.55436309954
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.5543630995404862, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.55)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: forward, reward: 1.07791999568
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.077919995679124, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.08)
65% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 244
\-------------------------
Environment.reset(): Trial set up with start = (7, 2), destination = (4, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0872; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 0.638503642718
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.6385036427184869, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 0.64)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 2.88575606634
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.8857560663390376, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.89)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: None, reward: 1.49187210216
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.4918721021568522, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.49)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: 1.39559888964
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 1.3955988896359948, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.40)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.40638764319
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.4063876431906062, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.41)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.13785627825
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.1378562782546364, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.14)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.29314638602
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.293146386019467, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.29)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: forward, reward: 1.73133969658
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.731339696584093, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.73)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.40900804243
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.409008042425572, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.41)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 2.02131940552
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 2.02131940552111, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.02)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.1165621094
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.116562109396397, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.12)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 2.57688793075
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.5768879307451678, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.58)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: None, reward: 0.933793947387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.9337939473872097, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.93)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: left, reward: 0.868248285649
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.8682482856491218, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.87)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 0.71274182899
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.712741828990298, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.71)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 2.15116806016
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 2.151168060157282, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.15)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: 0.928171360742
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'right', None), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.9281713607424971, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'right', None)
Agent drove right instead of forward. (rewarded 0.93)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: -0.0237639816178
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'right'), 'deadline': 3, 't': 17, 'action': 'forward', 'reward': -0.02376398161783555, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'right')
Agent drove forward instead of left. (rewarded -0.02)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 0.933760625849
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.9337606258486026, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 0.93)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: None, reward: 0.813325329017
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'right', None), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.8133253290169358, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 0.81)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 245
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (3, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0863; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0863; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0863; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0863; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0863; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0863; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0863; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: forward, reward: -9.72744958838
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'right'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'right', 'right', 'left'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': -9.72744958838002, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', 'left')
Agent attempted driving forward through a red light. (rewarded -9.73)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 1.36861161305
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.3686116130466266, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.37)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 1.99240444883
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.9924044488288275, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.99)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 2.44427557245
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.444275572452723, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.44)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 2.49572663602
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.4957266360168484, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.50)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 2.14984676774
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.1498467677419395, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.15)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 1.19544770334
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.1954477033421633, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.20)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 2.71074331526
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.7107433152550398, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.71)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 2.07828545922
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.0782854592204667, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 2.08)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 246
\-------------------------
Environment.reset(): Trial set up with start = (2, 7), destination = (7, 4), deadline = 30
Simulating trial. . .
epsilon = 0.0854; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0854; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0854; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: left, reward: 1.66587282424
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 1.6658728242379508, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.67)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 2.41176108692
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.4117610869229287, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.41)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 2.64610320868
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.646103208681827, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.65)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 1.99155556961
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 27, 't': 3, 'action': 'forward', 'reward': 1.991555569613844, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.99)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 2.5271423832
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 2.5271423832049082, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.53)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.8107696033
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 25, 't': 5, 'action': None, 'reward': 2.8107696033043967, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.81)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: None, reward: 2.87696515463
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.8769651546345085, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.88)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: left, reward: 2.5736882503
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 23, 't': 7, 'action': 'left', 'reward': 2.5736882502984466, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 2.57)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: right, reward: 0.164804635599
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 8, 'action': 'right', 'reward': 0.16480463559854563, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.16)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: left, reward: 2.39237252996
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 21, 't': 9, 'action': 'left', 'reward': 2.3923725299629774, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.39)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: 1.32644567249
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 1.3264456724886207, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.33)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 1.04973761364
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.0497376136438266, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.05)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: right, reward: 1.04868088862
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 1.0486808886193468, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.05)
57% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 247
\-------------------------
Environment.reset(): Trial set up with start = (3, 4), destination = (6, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0846; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: right, reward: 1.82618504805
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.8261850480502313, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'forward')
Agent drove right instead of forward. (rewarded 1.83)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 2.13721788199
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.137217881992604, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.14)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: left, reward: -9.92468612551
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': -9.924686125509215, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.92)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 1.49571099904
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.4957109990421782, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.50)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: 2.3824667237
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.3824667237030095, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.38)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: None, reward: -4.95782576129
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', None, None, 'right'), 'deadline': 20, 't': 5, 'action': None, 'reward': -4.95782576128974, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'right')
Agent idled at a green light with no oncoming traffic. (rewarded -4.96)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: right, reward: 1.06028475824
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.0602847582360693, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.06)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: left, reward: -40.2655694503
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'right'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', 'right'), 'deadline': 18, 't': 7, 'action': 'left', 'reward': -40.26556945033814, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.27)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: None, reward: 1.25608819241
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.256088192412472, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.26)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 1.35853623396
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.3585362339589762, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.36)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 1.92747289509
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.9274728950857294, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.93)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.72502225978
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.7250222597798377, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.73)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: left, reward: -9.90193280811
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 13, 't': 12, 'action': 'left', 'reward': -9.90193280811081, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -9.90)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: forward, reward: 1.13679321645
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 1.136793216445021, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.14)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 1.24123945191
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 1.2412394519114223, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.24)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: 1.19969456229
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', None), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 1.1996945622887532, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', None)
Agent followed the waypoint forward. (rewarded 1.20)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 1.89813958498
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.8981395849830551, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.90)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 4), heading: (0, -1), action: right, reward: 0.0948500087341
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 8, 't': 17, 'action': 'right', 'reward': 0.09485000873407734, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.09)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: right, reward: 1.31613323125
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 1.3161332312516294, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 1.32)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: None, reward: 0.576464814211
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 6, 't': 19, 'action': None, 'reward': 0.5764648142107547, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 0.58)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 1.96209787532
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 5, 't': 20, 'action': 'right', 'reward': 1.9620978753173615, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.96)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: right, reward: 0.749508051511
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 4, 't': 21, 'action': 'right', 'reward': 0.7495080515108887, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent followed the waypoint right. (rewarded 0.75)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 6), heading: (0, 1), action: left, reward: 1.51963462962
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 3, 't': 22, 'action': 'left', 'reward': 1.5196346296160719, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.52)
8% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 248
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (5, 7), deadline = 25
Simulating trial. . .
epsilon = 0.0837; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: -10.3466739566
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': -10.346673956628381, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent attempted driving forward through a red light. (rewarded -10.35)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 2.43206511444
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.4320651144429277, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.43)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: None, reward: 1.03580507189
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.0358050718939376, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.04)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 1.63494838386
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.634948383862393, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.63)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: forward, reward: 2.58679749144
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.5867974914449645, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.59)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: forward, reward: 1.66839008425
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.668390084247627, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.67)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: right, reward: 1.18383004046
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.1838300404620479, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.18)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: forward, reward: 1.62138508827
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.6213850882697107, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.62)
68% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 249
\-------------------------
Environment.reset(): Trial set up with start = (3, 2), destination = (6, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0829; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.22524406434
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.2252440643378746, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.23)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 1.30165523014
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.3016552301433806, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.30)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: None, reward: 2.72720608187
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.7272060818654102, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.73)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: forward, reward: 1.72053564412
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.720535644122597, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.72)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: 1.76582878872
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.7658287887192405, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.77)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 1.09059181786
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.0905918178577376, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.09)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 2.07438270561
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.0743827056108883, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.07)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: 1.64223965959
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'right'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.642239659589753, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'right')
Agent drove right instead of forward. (rewarded 1.64)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: left, reward: 1.27522922904
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.275229229042804, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', None)
Agent followed the waypoint left. (rewarded 1.28)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 2.6011395017
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.6011395017026695, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.60)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: 1.46877886273
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'forward'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 1.4687788627281235, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.47)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: 1.17264959422
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 1.1726495942248814, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', None)
Agent drove forward instead of left. (rewarded 1.17)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 0.753720672473
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.7537206724730052, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.75)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: left, reward: 1.76902842834
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.7690284283429045, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.77)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 0.612852761879
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 6, 't': 14, 'action': None, 'reward': 0.6128527618787067, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.61)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 1.09731527037
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.0973152703693874, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.10)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: None, reward: 1.40222321474
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.4022232147402602, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.40)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: left, reward: 1.46006400569
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 1.4600640056898946, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.46)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 1.57376849855
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.5737684985452953, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.57)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 0.505273491417
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.5052734914168155, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.51)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 250
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (7, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0821; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: left, reward: 2.79930988236
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 2.799309882355793, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.80)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 2.75583755746
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 2.7558375574632956, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.76)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.15483441399
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.1548344139949023, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.15)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.34186592974
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.3418659297388573, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.34)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.06335654152
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.0633565415231374, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.06)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: left, reward: 1.55504026311
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.5550402631143028, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent drove left instead of forward. (rewarded 1.56)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 1.82570740461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.8257074046139825, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.83)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.77415091542
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.774150915423755, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.77)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 2.34067344117
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.34067344116799, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.34)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 2.1190223955
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 2.1190223955043304, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.12)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.45012322178
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.4501232217837956, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.45)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: forward, reward: 1.39347517844
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.393475178441209, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 1.39)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: None, reward: 2.61623637775
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'right'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.616236377749518, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'right')
Agent properly idled at a red light. (rewarded 2.62)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: forward, reward: 1.00258228162
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 1.0025822816235088, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 1.00)
44% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 251
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (4, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0813; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 1.60103618412
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.6010361841154568, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', None)
Agent followed the waypoint right. (rewarded 1.60)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 2.81521345969
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.8152134596891116, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.82)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 2.61456380372
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.6145638037232644, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.61)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: -10.7747421746
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': -10.774742174590662, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -10.77)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: forward, reward: 1.68832325178
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 1.6883232517795457, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.69)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.84296117117
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.842961171174407, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.84)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.22326603401
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.223266034005953, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.22)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: forward, reward: 2.0687714182
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 2.0687714182038857, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.07)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: forward, reward: 1.85599104322
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 1.8559910432219817, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.86)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 0.159338398943
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 0.15933839894284652, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 0.16)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (-1, 0), action: right, reward: 2.45632745259
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 2.456327452591391, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.46)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 5), heading: (0, -1), action: right, reward: 0.885636946075
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 0.8856369460745293, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 0.89)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: 1.28133668297
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', None, None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.2813366829683295, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.28)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 2.34251217416
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 12, 't': 13, 'action': None, 'reward': 2.3425121741602677, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.34)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 1.43869358553
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 11, 't': 14, 'action': None, 'reward': 1.4386935855272485, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.44)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 1.71326088308
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 15, 'action': None, 'reward': 1.7132608830785128, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.71)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: left, reward: 1.94896641618
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 16, 'action': 'left', 'reward': 1.9489664161753577, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.95)
32% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 252
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (1, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0805; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0805; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0805; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0805; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 1.12915824674
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'right', 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.1291582467413253, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'right', 'left')
Agent drove right instead of forward. (rewarded 1.13)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: left, reward: 1.74228717586
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.7422871758647553, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.74)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 1.5669513267
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.5669513266972044, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 1.57)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.39808648419
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.3980864841932066, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.40)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.40373067078
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'right'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.403730670784861, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'right')
Agent properly idled at a red light. (rewarded 2.40)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: left, reward: 2.17417660929
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.1741766092878403, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.17)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 1.70000907207
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', 'forward'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.7000090720695984, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.70)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: 1.29015317627
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.2901531762733593, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'forward')
Agent followed the waypoint forward. (rewarded 1.29)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: 1.10813653265
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', 'forward'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 1.108136532650461, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', 'forward')
Agent drove forward instead of left. (rewarded 1.11)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 0.902600521184
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.9026005211842383, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.90)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 7), heading: (0, 1), action: left, reward: 0.784178538584
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', 'left'), 'deadline': 10, 't': 10, 'action': 'left', 'reward': 0.7841785385838336, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', 'left')
Agent followed the waypoint left. (rewarded 0.78)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: left, reward: 0.983262033378
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 0.9832620333778348, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.98)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 1.66879318763
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.6687931876265365, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 1.67)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 1.695695657
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.6956956569960369, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.70)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 2.18769739843
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': 2.187697398425858, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.19)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 0.745405019994
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.7454050199942788, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.75)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 1.71556743263
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.7155674326346662, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.72)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: left, reward: -19.2083401842
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': -19.208340184218375, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -19.21)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: forward, reward: 0.248562092554
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 2, 't': 18, 'action': 'forward', 'reward': 0.2485620925537042, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 0.25)
5% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 253
\-------------------------
Environment.reset(): Trial set up with start = (1, 7), destination = (5, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0797; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0797; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0797; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: left, reward: 1.60568899336
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'left'), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 1.6056889933570477, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'left')
Agent followed the waypoint left. (rewarded 1.61)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 1.08168281268
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'forward'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.0816828126796518, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 1.08)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: left, reward: 2.0480116243
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 2.0480116243009725, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.05)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 2.34453152448
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.3445315244760443, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.34)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: forward, reward: 1.93810015385
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 1.9381001538543126, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.94)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: forward, reward: 1.52005705173
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.5200570517294718, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 1.52)
76% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 254
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (5, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0789; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0789; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0789; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.73648889367
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'forward'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.7364888936739, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'forward')
Agent properly idled at a red light. (rewarded 1.74)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.16718252346
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.167182523463601, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.17)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.64180941482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.6418094148247753, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.64)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.2977894153
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.297789415295111, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.30)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: left, reward: 1.31252773597
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 1.3125277359745473, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.31)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: 2.56401016814
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 2.564010168142989, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 2.56)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 1.67364588037
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.6736458803749796, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.67)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.63126093461
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.6312609346127642, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.63)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: -10.1669946799
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': -10.16699467994068, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent attempted driving forward through a red light. (rewarded -10.17)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 1.99895096748
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.9989509674818442, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.00)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 1.30026327581
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.3002632758148427, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.30)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: 1.12377404462
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.123774044617858, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.12)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.9206030255
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.9206030255043376, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.92)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: 1.46392357169
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 1.463923571688928, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.46)
44% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 255
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (5, 6), deadline = 30
Simulating trial. . .
epsilon = 0.0781; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: 2.20413179348
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 2.2041317934758355, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.20)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 2.57923717623
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 29, 't': 1, 'action': 'forward', 'reward': 2.5792371762342654, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent followed the waypoint forward. (rewarded 2.58)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (0, 1), action: left, reward: 1.12030483773
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'left'), 'deadline': 28, 't': 2, 'action': 'left', 'reward': 1.1203048377298526, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'left')
Agent drove left instead of forward. (rewarded 1.12)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: right, reward: 1.44764180242
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 27, 't': 3, 'action': 'right', 'reward': 1.4476418024154254, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 1.45)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: forward, reward: 2.89284246791
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 26, 't': 4, 'action': 'forward', 'reward': 2.8928424679054823, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.89)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: forward, reward: 0.197178227916
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, 'right'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 0.19717822791602646, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, 'right')
Agent drove forward instead of right. (rewarded 0.20)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 2.90530206955
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 24, 't': 6, 'action': 'right', 'reward': 2.9053020695484086, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.91)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: 2.83803926594
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 23, 't': 7, 'action': 'right', 'reward': 2.838039265942297, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 2.84)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: left, reward: 1.42855197139
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 22, 't': 8, 'action': 'left', 'reward': 1.4285519713935326, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.43)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: forward, reward: 1.74326720754
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 1.7432672075364535, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.74)
67% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 256
\-------------------------
Environment.reset(): Trial set up with start = (5, 7), destination = (8, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0773; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.18555928263
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.1855592826300656, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.19)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.954342475
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.9543424749971658, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 1.95)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.77407251554
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.7740725155437778, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.77)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: 2.86231614947
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 2.862316149468107, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.86)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: 1.49547252713
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.4954725271256646, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.50)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: right, reward: 0.414658315418
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.41465831541819564, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.41)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 2.85071027852
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.850710278517889, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.85)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: forward, reward: 1.53441442372
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.5344144237205364, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 1.53)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: right, reward: 1.38877346019
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'right'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.3887734601860418, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'right')
Agent drove right instead of forward. (rewarded 1.39)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: left, reward: 1.84341290271
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 1.8434129027112862, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.84)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: right, reward: 1.41059837084
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.4105983708355954, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', None)
Agent drove right instead of forward. (rewarded 1.41)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: 0.545792900218
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 14, 't': 11, 'action': 'right', 'reward': 0.5457929002175684, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove right instead of left. (rewarded 0.55)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.82690033286
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 1.8269003328608189, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.83)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (-1, 0), action: right, reward: 1.31472747134
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'left', None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.3147274713424013, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 1.31)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: forward, reward: 0.815907528169
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 11, 't': 14, 'action': 'forward', 'reward': 0.8159075281690273, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 0.82)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 2.11308334535
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 10, 't': 15, 'action': 'forward', 'reward': 2.113083345347529, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.11)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.45241913633
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.452419136328567, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.45)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 0.722604933177
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 17, 'action': None, 'reward': 0.7226049331770492, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.72)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.63120829669
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': None, 'reward': 1.6312082966946935, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.63)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 0.739485864324
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 6, 't': 19, 'action': None, 'reward': 0.739485864323812, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 0.74)
20% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.60956116411
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 5, 't': 20, 'action': None, 'reward': 1.609561164108021, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.61)
16% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: left, reward: 2.00151218317
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 4, 't': 21, 'action': 'left', 'reward': 2.0015121831683693, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.00)
12% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (0, 1), action: forward, reward: 1.68624496548
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 3, 't': 22, 'action': 'forward', 'reward': 1.6862449654803913, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent followed the waypoint forward. (rewarded 1.69)
8% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 257
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (5, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0765; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0765; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 1.18732062402
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.1873206240181107, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.19)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 1.80292499241
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.802924992405357, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.80)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: forward, reward: -39.6138835244
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'right'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'right', 'forward', 'left'), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': -39.6138835244231, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'forward', 'left')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -39.61)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 2.07500666995
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.0750066699485386, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.08)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 1.91231290315
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.9123129031483148, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.91)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (0, -1), action: None, reward: 1.58156848121
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.5815684812144781, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.58)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: left, reward: 2.9065524509
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 2.906552450902006, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 2.91)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 1.50544121289
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.5054412128927948, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.51)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 4), heading: (-1, 0), action: forward, reward: 0.987777482485
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 0.9877774824845715, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 0.99)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: forward, reward: 2.45081745479
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.450817454790947, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.45)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.01012721745
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.0101272174528857, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.01)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.58288887163
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.5828888716275147, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.58)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: None, reward: 1.76293135013
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 13, 't': 12, 'action': None, 'reward': 1.7629313501287969, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.76)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: right, reward: 1.59153436758
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 12, 't': 13, 'action': 'right', 'reward': 1.5915343675757607, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded 1.59)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 1.41067041781
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 1.410670417810603, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.41)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: None, reward: 2.65567421405
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 10, 't': 15, 'action': None, 'reward': 2.655674214049596, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.66)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: right, reward: 2.25247630331
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 9, 't': 16, 'action': 'right', 'reward': 2.2524763033064383, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 2.25)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 4), heading: (0, 1), action: None, reward: 0.61041761643
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 8, 't': 17, 'action': None, 'reward': 0.6104176164302533, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 0.61)
28% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 4), heading: (-1, 0), action: right, reward: 2.26159631309
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 7, 't': 18, 'action': 'right', 'reward': 2.2615963130879324, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.26)
24% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (0, 1), action: left, reward: 2.37830812694
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 6, 't': 19, 'action': 'left', 'reward': 2.378308126936245, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.38)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 258
\-------------------------
Environment.reset(): Trial set up with start = (2, 6), destination = (8, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0758; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 1.18867891922
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.1886789192203555, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.19)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.33443627176
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.3344362717604483, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.33)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 2.88773886405
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.8877388640503834, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.89)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: None, reward: 1.48697900005
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.4869790000540757, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.49)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: right, reward: 2.90869414742
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 2.908694147421417, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 2.91)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 1.50114502017
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.501145020167799, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.50)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 1.98322318157
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'right', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.983223181565349, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'right', None)
Agent properly idled at a red light. (rewarded 1.98)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 2.09098784901
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.090987849006042, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.09)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: right, reward: 1.03352885534
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.0335288553351583, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded 1.03)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 2.77441394008
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.7744139400774337, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.77)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 1.57953452635
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.5795345263465266, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.58)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: left, reward: 1.69381553981
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'left', 'reward': 1.6938155398078214, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 1.69)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.44280040316
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.4428004031577124, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.44)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 0.71728230427
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 0.7172823042702109, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded 0.72)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: right, reward: 2.55337364085
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 2.5533736408476315, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.55)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: right, reward: 1.13302128724
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 1.133021287240628, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 1.13)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 0.523342544747
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 0.5233425447467819, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent drove right instead of forward. (rewarded 0.52)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.20639378982
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.2063937898203072, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.21)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: None, reward: 1.36794211997
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 1.367942119970483, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.37)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: -0.629160978634
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', 'right'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': -0.6291609786344444, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', 'forward', 'right', 'right')
Agent drove right instead of left. (rewarded -0.63)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 259
\-------------------------
Environment.reset(): Trial set up with start = (2, 2), destination = (8, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0750; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: 2.00370610491
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.0037061049103917, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.00)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: 1.98340939007
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'right'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.9834093900721714, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'right')
Agent followed the waypoint forward. (rewarded 1.98)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 1.35747140579
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.3574714057866077, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.36)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.69287276424
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.6928727642422663, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.69)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.50686365505
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.506863655049034, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.51)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.88571868844
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.8857186884430837, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.89)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 0.00279214835686
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 0.0027921483568638106, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.00)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: forward, reward: 2.7687476592
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.768747659201737, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.77)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 260
\-------------------------
Environment.reset(): Trial set up with start = (5, 4), destination = (1, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0743; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0743; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0743; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: forward, reward: 1.90760018517
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 25, 't': 0, 'action': 'forward', 'reward': 1.9076001851695614, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 1.91)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 2.02188782076
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.0218878207633253, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.02)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.60877716942
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.608777169415042, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 1.61)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 2.89001723016
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.8900172301639193, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.89)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: -9.7991510025
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': -9.799151002503066, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.80)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: right, reward: 0.470348685382
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', 'right'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 0.47034868538222974, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', 'right')
Agent drove right instead of forward. (rewarded 0.47)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: 1.25481023035
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.2548102303482784, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.25)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.45603490907
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.4560349090661147, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.46)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.92042517138
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.9204251713845404, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.92)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: forward, reward: 1.50914873762
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'forward'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.5091487376232324, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'forward')
Agent followed the waypoint forward. (rewarded 1.51)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 261
\-------------------------
Environment.reset(): Trial set up with start = (1, 5), destination = (7, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0735; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: left, reward: 1.93981638339
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.939816383389057, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.94)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: right, reward: 1.7005069698
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.7005069697968709, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, None)
Agent followed the waypoint right. (rewarded 1.70)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.4881656283
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.4881656282953803, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.49)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.10898383535
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.1089838353508277, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.11)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 1.19791370827
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.197913708274694, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.20)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: -0.0458899050808
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': -0.0458899050807855, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove right instead of left. (rewarded -0.05)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 2.49206391429
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.4920639142856036, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.49)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.74144848098
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.74144848098098, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.74)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.80145184145
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'forward', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.8014518414498062, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 1.80)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: right, reward: 2.55993542218
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 2.5599354221824897, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.56)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 1.72629223154
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.726292231538385, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.73)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: None, reward: 2.40869167056
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.4086916705619186, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.41)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: right, reward: 0.807409572016
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'forward'), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.8074095720164334, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'forward')
Agent followed the waypoint right. (rewarded 0.81)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 0.667832389319
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'left', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 0.6678323893188631, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 0.67)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.62478833707
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'right'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.6247883370681933, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'right')
Agent properly idled at a red light. (rewarded 1.62)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: None, reward: 1.50874530369
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.5087453036883411, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.51)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: left, reward: 1.54490822534
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 1.5449082253371005, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.54)
15% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 262
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (5, 7), deadline = 25
Simulating trial. . .
epsilon = 0.0728; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: left, reward: 1.53358748176
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 1.5335874817576793, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.53)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 2.87011451196
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.870114511955225, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.87)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 6), heading: (0, 1), action: None, reward: 2.176737252
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.1767372519968258, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.18)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: left, reward: 1.42865696113
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'left', 'reward': 1.4286569611329472, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.43)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 7), heading: (0, 1), action: right, reward: 1.91745607261
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'forward'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.917456072612607, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.92)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 2), heading: (0, 1), action: forward, reward: 1.54751346443
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.5475134644344242, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded 1.55)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: right, reward: 0.62967629183
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'right'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 0.6296762918300419, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'right')
Agent drove right instead of left. (rewarded 0.63)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: right, reward: 2.38511701752
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 2.3851170175241485, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', None)
Agent followed the waypoint right. (rewarded 2.39)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 1.69785732793
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, 'forward'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.6978573279299387, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, 'forward')
Agent followed the waypoint right. (rewarded 1.70)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: 1.75111461225
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.7511146122474943, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 1.75)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 1.67674045992
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.676740459915746, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.68)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.04405148626
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'forward'), 'deadline': 14, 't': 11, 'action': None, 'reward': 2.044051486256474, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'forward')
Agent properly idled at a red light. (rewarded 2.04)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.72495482718
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', 'forward'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.7249548271764747, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', 'forward')
Agent properly idled at a red light. (rewarded 2.72)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: forward, reward: 2.00679173925
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 2.006791739245849, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.01)
44% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 263
\-------------------------
Environment.reset(): Trial set up with start = (2, 2), destination = (6, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0721; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: right, reward: 2.59408951842
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'right', 'left'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.5940895184172232, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'right', 'left')
Agent followed the waypoint right. (rewarded 2.59)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: right, reward: 2.63624082113
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.6362408211260115, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.64)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.32857758326
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.3285775832560027, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.33)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.11109082956
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.1110908295576711, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.11)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 1.16016320337
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.1601632033704008, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', None, None)
Agent followed the waypoint forward. (rewarded 1.16)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 2.87747091694
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 2.8774709169426345, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.88)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: forward, reward: 2.05157002013
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 2.0515700201307707, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.05)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (-1, 0), action: None, reward: 2.77797738087
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.777977380872684, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.78)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 2), heading: (0, -1), action: right, reward: 2.09644986597
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.0964498659700075, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.10)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 264
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (8, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0714; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.62515299611
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', 'right', 'forward'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.625152996105678, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', 'right', 'forward')
Agent properly idled at a red light. (rewarded 2.63)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 2.61610061105
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.6161006110483935, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.62)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: -40.9641148501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 4, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 18, 't': 2, 'action': 'left', 'reward': -40.964114850052134, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -40.96)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: None, reward: 1.35588381363
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.3558838136258657, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.36)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 6), heading: (0, -1), action: right, reward: 1.3035101672
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.3035101671986702, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.30)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: right, reward: 2.73378647865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'right', None), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 2.7337864786460164, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'right', None)
Agent followed the waypoint right. (rewarded 2.73)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 1.83423602916
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.8342360291636142, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.83)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: 1.81731843693
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.8173184369297142, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 1.82)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: left, reward: 2.72094361718
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 2.7209436171757293, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.72)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 265
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (4, 7), deadline = 25
Simulating trial. . .
epsilon = 0.0707; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 1.14139547995
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.1413954799503012, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.14)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 1.56099396826
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.5609939682574447, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.56)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: left, reward: -9.52726767818
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 23, 't': 2, 'action': 'left', 'reward': -9.52726767817786, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent attempted driving left through a red light. (rewarded -9.53)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 6), heading: (0, -1), action: None, reward: 2.46941114233
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.469411142326373, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.47)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: right, reward: 1.97687296895
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.976872968945495, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.98)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.19955120255
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.1995512025492248, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.20)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.68554577917
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.6855457791733026, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.69)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.11488155389
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.1148815538912493, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.11)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.5229582386
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.5229582386024338, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.52)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 2.81178303711
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.811783037111174, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.81)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: forward, reward: 0.908436918223
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 0.9084369182226186, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 0.91)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 1.97563204351
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.9756320435115693, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.98)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: None, reward: 2.36203223401
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.3620322340134434, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.36)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: forward, reward: 1.67510093377
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 1.675100933770409, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.68)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 2.26110921917
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 2.2611092191687465, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.26)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 266
\-------------------------
Environment.reset(): Trial set up with start = (3, 7), destination = (5, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0699; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 2.57993846672
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 2.5799384667150234, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 2.58)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.6446889331
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.644688933095127, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.64)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 1.41368422125
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.4136842212450562, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.41)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.3380907369
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.3380907369001998, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.34)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.77427006721
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.7742700672129272, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 2.77)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 1.08636071445
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.086360714445894, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove right instead of forward. (rewarded 1.09)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: left, reward: -10.9673096822
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': -10.967309682165936, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent attempted driving left through a red light. (rewarded -10.97)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: right, reward: 1.86016115781
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.8601611578051478, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'forward')
Agent drove right instead of left. (rewarded 1.86)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: None, reward: 2.78787602605
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.787876026045125, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.79)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: right, reward: 1.76005088041
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.7600508804052688, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.76)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 1.95634548694
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.956345486942016, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.96)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 1.79327061387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 1.7932706138668948, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.79)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 2.05598042373
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.0559804237260546, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.06)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: forward, reward: 1.4008310562
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.4008310562013049, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.40)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: left, reward: 1.90846448296
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 6, 't': 14, 'action': 'left', 'reward': 1.9084644829568183, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 1.91)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: forward, reward: 2.17181328209
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 5, 't': 15, 'action': 'forward', 'reward': 2.171813282092086, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 2.17)
20% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 267
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (5, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0693; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0693; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0693; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: left, reward: -39.2652317141
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'forward'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -39.26523171413189, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'forward')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.27)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: right, reward: 0.947437684257
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.9474376842566935, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.95)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 2.56998963361
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.5699896336145, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.57)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.96194400695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.9619440069459428, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.96)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.7680443502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.7680443501970207, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.77)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (0, -1), action: None, reward: 1.27428643227
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.2742864322718537, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.27)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: left, reward: 2.23578183501
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.235781835011209, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 2.24)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: right, reward: 1.13556933397
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'left'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.135569333967169, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'left')
Agent drove right instead of forward. (rewarded 1.14)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (6, 6), heading: (-1, 0), action: left, reward: 1.1400935462
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'right', None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.1400935461963408, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'right', None)
Agent followed the waypoint left. (rewarded 1.14)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 6), heading: (-1, 0), action: forward, reward: 1.65953284945
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.659532849454077, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.66)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: right, reward: 1.43954787055
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.4395478705478344, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.44)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: None, reward: 2.56421588911
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.5642158891051903, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.56)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: None, reward: 1.2776794652
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.2776794651976162, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.28)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: None, reward: 2.46796203107
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': None, 'reward': 2.4679620310708166, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.47)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: None, reward: 1.79653115987
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.7965311598656262, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.80)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (5, 5), heading: (0, -1), action: None, reward: 0.892604738381
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.8926047383811906, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.89)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: forward, reward: 1.89488866048
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 4, 't': 16, 'action': 'forward', 'reward': 1.8948886604798132, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.89)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: left, reward: -9.58685869612
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': 'left', 'reward': -9.58685869612014, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -9.59)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 2.21813686635
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 2, 't': 18, 'action': None, 'reward': 2.21813686634962, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.22)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: forward, reward: 1.62088960574
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 1, 't': 19, 'action': 'forward', 'reward': 1.6208896057440239, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.62)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 268
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (7, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0686; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: 1.64487536703
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.644875367028644, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 1.64)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: right, reward: 0.239378997662
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 0.2393789976619286, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'left')
Agent drove right instead of forward. (rewarded 0.24)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 1.85260357234
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.852603572341161, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.85)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 3), heading: (0, 1), action: None, reward: 2.51173560388
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.5117356038774457, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.51)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: right, reward: 1.64222521776
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 1.6422252177574563, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove right instead of left. (rewarded 1.64)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: None, reward: 2.13655384059
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'right', None), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.136553840594711, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.14)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: right, reward: 2.41898424129
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 2.4189842412902127, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.42)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: right, reward: 1.08608633647
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.086086336471894, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', None)
Agent followed the waypoint right. (rewarded 1.09)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: None, reward: 1.12738271133
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.1273827113336143, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.13)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 1.99680739905
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'forward', 'reward': 1.9968073990468473, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.00)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: -9.20679461549
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': -9.206794615486746, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent attempted driving forward through a red light. (rewarded -9.21)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.15571402124
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.155714021236422, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.16)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 0.667135412189
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 0.6671354121892492, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent drove right instead of forward. (rewarded 0.67)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.94957200451
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 7, 't': 13, 'action': None, 'reward': 1.9495720045136042, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.95)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 1.00181942801
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.0018194280136723, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.00)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 0.840239973739
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 5, 't': 15, 'action': None, 'reward': 0.840239973738889, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 0.84)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: 1.8851863219
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 4, 't': 16, 'action': 'left', 'reward': 1.8851863219027618, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.89)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 1.54492718991
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.5449271899146324, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.54)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 0.898874585067
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.898874585067011, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.90)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: left, reward: 0.693359195249
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 1, 't': 19, 'action': 'left', 'reward': 0.6933591952492648, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.69)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 269
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (4, 7), deadline = 30
Simulating trial. . .
epsilon = 0.0679; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: left, reward: 1.49798617474
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 1.4979861747429446, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 1.50)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: right, reward: 0.994691514027
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 0.994691514027003, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', None)
Agent followed the waypoint right. (rewarded 0.99)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 1.22514987444
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 28, 't': 2, 'action': 'forward', 'reward': 1.2251498744378977, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.23)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 0.336566611672
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'forward', None), 'deadline': 27, 't': 3, 'action': 'left', 'reward': 0.3365666116718088, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'forward', None)
Agent drove left instead of forward. (rewarded 0.34)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.60656731367
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.606567313667118, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.61)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 1.08537053976
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 25, 't': 5, 'action': 'right', 'reward': 1.0853705397586526, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 1.09)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: 1.47853302207
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 1.478533022068052, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.48)
77% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 270
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (3, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0672; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (0, -1), action: right, reward: 2.68972814372
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'right'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.689728143717709, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'right')
Agent followed the waypoint right. (rewarded 2.69)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: forward, reward: 1.8486908736
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 1.8486908736042946, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove forward instead of right. (rewarded 1.85)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: None, reward: 2.10114053265
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.1011405326454233, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.10)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 1.31827634615
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 1.318276346147555, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.32)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: None, reward: 2.44781799792
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.4478179979227255, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.45)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: left, reward: 0.49654043424
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'left'), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 0.49654043424031935, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'left')
Agent drove left instead of forward. (rewarded 0.50)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 6), heading: (1, 0), action: right, reward: 1.42126614556
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.4212661455567621, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', None)
Agent followed the waypoint right. (rewarded 1.42)
72% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 271
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (8, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0665; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: left, reward: -39.610832213
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', 'right'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': -39.61083221298417, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', 'right')
Agent attempted driving left through a red light with traffic and cause a major accident. (rewarded -39.61)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 1.29721910644
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.2972191064392071, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.30)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 1.112952174
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.11295217400448, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.11)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 7), heading: (0, 1), action: None, reward: 1.20398843655
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.2039884365493105, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.20)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: right, reward: 0.125579623732
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 16, 't': 4, 'action': 'right', 'reward': 0.1255796237318515, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 0.13)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 1.10455087764
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.1045508776372783, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.10)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: None, reward: 2.33753871393
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.3375387139332857, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.34)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: left, reward: 1.92724831089
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.9272483108914673, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.93)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 2), heading: (0, 1), action: None, reward: 2.45048440628
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.450484406279645, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.45)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: right, reward: 1.00534677401
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.0053467740123374, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.01)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 2), heading: (-1, 0), action: None, reward: 2.22967148456
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.2296714845583776, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.23)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: right, reward: 1.13588331794
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.1358833179362078, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', None)
Agent drove right instead of forward. (rewarded 1.14)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (0, -1), action: None, reward: 1.25563095695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.2556309569495807, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 1.26)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: left, reward: 1.4687188091
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.4687188091027856, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.47)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: -0.289545583851
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', 'right'), 'deadline': 6, 't': 14, 'action': 'right', 'reward': -0.28954558385098095, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', 'right')
Agent drove right instead of forward. (rewarded -0.29)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.64576418113
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.6457641811349222, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.65)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 1.76118640405
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', None, 'forward'), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.7611864040543028, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', None, 'forward')
Agent properly idled at a red light. (rewarded 1.76)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 0.57054228804
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'forward'), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 0.5705422880395924, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'forward')
Agent drove right instead of left. (rewarded 0.57)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 0.769586745143
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'right'), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.7695867451426448, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 0.77)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 1.59787330479
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'right'), 'deadline': 1, 't': 19, 'action': 'right', 'reward': 1.597873304785767, 'waypoint': 'right'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('right', 'green', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.60)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 272
\-------------------------
Environment.reset(): Trial set up with start = (3, 3), destination = (2, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0659; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: None, reward: 2.33380133521
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.3338013352119784, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.33)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: None, reward: 1.65225764799
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.652257647986735, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.65)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: None, reward: 2.68081557635
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.680815576353922, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.68)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: None, reward: 2.10807982232
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.1080798223234742, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.11)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: None, reward: 2.22234177857
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.2223417785677766, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.22)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 3), heading: (0, -1), action: None, reward: 2.70922273499
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.709222734993289, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.71)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (-1, 0), action: left, reward: 2.15921056594
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 2.1592105659414385, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.16)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 1.11043257107
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.110432571066036, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.11)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: left, reward: 1.41715046932
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 1.417150469321681, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove left instead of forward. (rewarded 1.42)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 2.53343620075
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 2.5334362007505016, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.53)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 1.20599634695
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.2059963469507142, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.21)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 0.774499420651
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'forward', None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.7744994206511613, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'forward', None)
Agent drove right instead of left. (rewarded 0.77)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: None, reward: 2.13850805129
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.138508051290155, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 2.14)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: 1.40433138357
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', None), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 1.4043313835677107, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', None)
Agent followed the waypoint right. (rewarded 1.40)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 1.97015586497
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 6, 't': 14, 'action': None, 'reward': 1.9701558649677018, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.97)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 1.98504880347
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.9850488034678613, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.99)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 2.19076524766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 4, 't': 16, 'action': 'right', 'reward': 2.1907652476638124, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 2.19)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (2, 7), heading: (1, 0), action: right, reward: 2.2249178919
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'right', None, None), 'deadline': 3, 't': 17, 'action': 'right', 'reward': 2.224917891897314, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'right', None, None)
Agent followed the waypoint right. (rewarded 2.22)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 6), heading: (0, -1), action: left, reward: 1.57504271017
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 2, 't': 18, 'action': 'left', 'reward': 1.575042710171257, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.58)
5% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 273
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (3, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0652; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 1.10662402658
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.1066240265815488, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.11)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 1.47780808442
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 1.4778080844175947, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.48)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.20750259862
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.2075025986248187, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.21)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 2.12132343967
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.121323439670645, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.12)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.78921090796
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.78921090796344, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.79)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 1.59494139502
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.5949413950177027, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.59)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 2.32365372797
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 2.3236537279701936, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 2.32)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: right, reward: 1.37459365746
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 1.3745936574584647, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove right instead of forward. (rewarded 1.37)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: None, reward: 1.21583531821
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.215835318210654, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 1.22)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: left, reward: 0.876409010205
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 0.876409010205162, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 0.88)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (2, 5), heading: (0, 1), action: None, reward: 1.16616151937
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'right'), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.1661615193656292, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'right')
Agent properly idled at a red light. (rewarded 1.17)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: left, reward: 1.62057311498
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'left', 'reward': 1.620573114982539, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.62)
52% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 274
\-------------------------
Environment.reset(): Trial set up with start = (4, 3), destination = (6, 7), deadline = 20
Simulating trial. . .
epsilon = 0.0646; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 3), heading: (-1, 0), action: right, reward: 0.623693132367
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.6236931323673836, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 0.62)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 2), heading: (0, -1), action: right, reward: 2.23286936246
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.2328693624579676, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.23)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: right, reward: 2.53308662238
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', 'left'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 2.533086622375101, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 2.53)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 2), heading: (1, 0), action: None, reward: 1.16616907706
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.166169077064742, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.17)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 2), heading: (1, 0), action: forward, reward: 1.11332822207
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.1133282220746021, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.11)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: forward, reward: 2.39504555489
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 2.395045554890495, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.40)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: -5.51847815373
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'left'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'left', 'right', 'forward'), 'deadline': 14, 't': 6, 'action': None, 'reward': -5.518478153730996, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'right', 'forward')
Agent idled at a green light with no oncoming traffic. (rewarded -5.52)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 0.735888679673
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, 'right'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 0.7358886796728581, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, 'right')
Agent drove right instead of left. (rewarded 0.74)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 3), heading: (-1, 0), action: right, reward: 1.42274609695
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.4227460969466785, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.42)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: right, reward: 1.4713234325
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 1.4713234325020297, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.47)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 2), heading: (0, -1), action: None, reward: 1.53074543534
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.5307454353422427, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.53)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: right, reward: 1.7282543731
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 1.728254373095088, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.73)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 0.799433739459
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 0.7994337394585591, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.80)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: left, reward: 2.10994262866
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 2.1099426286570795, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.11)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 275
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (8, 2), deadline = 35
Simulating trial. . .
epsilon = 0.0639; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 0.0600829484595
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', None), 'deadline': 35, 't': 0, 'action': 'right', 'reward': 0.060082948459481145, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', None)
Agent drove right instead of forward. (rewarded 0.06)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 2.13314736239
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 34, 't': 1, 'action': None, 'reward': 2.133147362389966, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.13)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 1.7912043777
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 33, 't': 2, 'action': None, 'reward': 1.7912043777031836, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.79)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: None, reward: 2.16415338929
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 32, 't': 3, 'action': None, 'reward': 2.164153389285504, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.16)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: left, reward: 2.2543516118
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 31, 't': 4, 'action': 'left', 'reward': 2.2543516118022806, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.25)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 2.51440893143
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 30, 't': 5, 'action': 'forward', 'reward': 2.5144089314276914, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.51)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (0, -1), action: right, reward: -0.0184733658371
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 29, 't': 6, 'action': 'right', 'reward': -0.018473365837089473, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent drove right instead of forward. (rewarded -0.02)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: left, reward: 1.3097868128
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 28, 't': 7, 'action': 'left', 'reward': 1.3097868128006391, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.31)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.32917853912
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 27, 't': 8, 'action': None, 'reward': 1.3291785391155733, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.33)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 3), heading: (-1, 0), action: None, reward: 1.26050021502
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 26, 't': 9, 'action': None, 'reward': 1.26050021502456, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.26)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (-1, 0), action: forward, reward: 1.56297047831
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 25, 't': 10, 'action': 'forward', 'reward': 1.5629704783073346, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.56)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 3), heading: (-1, 0), action: forward, reward: 1.44250736603
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'right', None), 'deadline': 24, 't': 11, 'action': 'forward', 'reward': 1.4425073660349024, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'right', None)
Agent drove forward instead of right. (rewarded 1.44)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (7, 2), heading: (0, -1), action: right, reward: 1.28806762197
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'forward'), 'deadline': 23, 't': 12, 'action': 'right', 'reward': 1.2880676219731741, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 1.29)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: right, reward: 1.88673280582
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 22, 't': 13, 'action': 'right', 'reward': 1.886732805822914, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.89)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 276
\-------------------------
Environment.reset(): Trial set up with start = (4, 2), destination = (1, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0633; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 2), heading: (-1, 0), action: left, reward: 2.04211758738
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.0421175873818846, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 2.04)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 2), heading: (-1, 0), action: forward, reward: 1.17007356479
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'right'), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.1700735647933194, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'right')
Agent followed the waypoint forward. (rewarded 1.17)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: forward, reward: 1.84834109504
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 1.848341095043711, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.85)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: left, reward: 2.50548906895
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 2.5054890689492506, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', None, None)
Agent followed the waypoint left. (rewarded 2.51)
80% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 277
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (4, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0627; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.51677716843
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.516777168428109, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.52)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.94465824523
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.944658245231005, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.94)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 2.43901824267
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.439018242667677, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.44)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.63744345532
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.6374434553155273, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.64)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.08771225665
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.0877122566506068, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.09)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 1.64455394407
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.6445539440720167, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.64)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 7), heading: (-1, 0), action: left, reward: 1.43693739245
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 19, 't': 6, 'action': 'left', 'reward': 1.436937392449127, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.44)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: forward, reward: 1.65516480922
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 1.6551648092210103, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.66)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: -10.9062816332
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 2, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 8, 'action': 'left', 'reward': -10.906281633215933, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent attempted driving left through a red light. (rewarded -10.91)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 6), heading: (0, -1), action: right, reward: 1.26860602446
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'right', 'reward': 1.268606024463041, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.27)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 5), heading: (0, -1), action: forward, reward: 2.5984786556
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 2.598478655603727, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.60)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 278
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (8, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0620; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.15381956372
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.15381956372431, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.15)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.98742421714
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.987424217144956, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.99)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 1.38189384504
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.3818938450374016, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.38)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 1.91770595435
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'forward', 'reward': 1.917705954351822, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.92)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 2.14182407828
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.1418240782847136, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.14)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 1.08823466997
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': None, 'reward': 1.0882346699674528, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.09)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: 1.25805565684
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'forward'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.2580556568369015, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded 1.26)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: 1.44148355638
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'forward', 'forward'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.4414835563811799, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.44)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 2.5886427175
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.588642717498345, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.59)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 0.851040603995
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 0.851040603995272, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.85)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 2.6119516101
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.6119516100975986, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.61)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: right, reward: 0.865090023451
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 9, 't': 11, 'action': 'right', 'reward': 0.8650900234512435, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 0.87)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: None, reward: 1.20145311192
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.20145311191764, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.20)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 2.04512591766
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 2.045125917663078, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 2.05)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 1.4277659726
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 1.4277659725992151, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.43)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: right, reward: 0.636106379594
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', None), 'deadline': 5, 't': 15, 'action': 'right', 'reward': 0.636106379594487, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', None)
Agent drove right instead of forward. (rewarded 0.64)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.27116562032
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.2711656203186184, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.27)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 1.92385459726
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 3, 't': 17, 'action': None, 'reward': 1.9238545972633114, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.92)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 0.976770872758
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 2, 't': 18, 'action': None, 'reward': 0.976770872758393, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 0.98)
5% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: None, reward: 0.743391799766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 1, 't': 19, 'action': None, 'reward': 0.7433917997662394, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.74)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 279
\-------------------------
Environment.reset(): Trial set up with start = (8, 4), destination = (3, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0614; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 2.05633773178
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.0563377317837004, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.06)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.51221356789
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.5122135678859436, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.51)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 2.47636611605
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.476366116045297, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.48)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 4), heading: (0, 1), action: None, reward: 1.89790136415
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.8979013641453828, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.90)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: left, reward: 2.39393807621
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'left', 'reward': 2.3939380762064006, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.39)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: right, reward: 1.44837864596
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'forward'), 'deadline': 20, 't': 5, 'action': 'right', 'reward': 1.448378645961252, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'forward')
Agent drove right instead of forward. (rewarded 1.45)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: None, reward: 2.49404752806
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 2.494047528061305, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.49)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: left, reward: 1.40817938462
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 1.4081793846172563, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.41)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: None, reward: 1.18438523714
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.1843852371359982, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.18)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: forward, reward: 2.38748415422
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.387484154215399, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.39)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 2.61059172071
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 2.610591720707886, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.61)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 280
\-------------------------
Environment.reset(): Trial set up with start = (8, 7), destination = (3, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0608; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 7), heading: (1, 0), action: right, reward: 2.80232209717
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.802322097174313, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.80)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: right, reward: 0.995678036173
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'forward'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 0.9956780361727327, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.00)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 2.87003883636
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 2.870038836357578, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.87)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 2.08542565183
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.08542565183474, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.09)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: None, reward: 2.47887754067
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.4788775406680625, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.48)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: left, reward: 1.51386051139
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.5138605113930188, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.51)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.62157999636
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.6215799963580633, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.62)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 2.75820622773
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.7582062277343287, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.76)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.81967163972
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.8196716397213268, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.82)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.69297045082
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.6929704508212644, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.69)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 3), heading: (0, 1), action: right, reward: 2.31396313699
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 2.313963136991852, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.31)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 281
\-------------------------
Environment.reset(): Trial set up with start = (6, 4), destination = (1, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0602; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.81521352054
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.81521352054353, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.82)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.44487151759
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.4448715175898519, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 1.44)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 2.02220506729
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.022205067289093, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.02)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: None, reward: 1.00997056009
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.0099705600927522, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.01)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 2.80383324277
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 2.803833242765201, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 2.80)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: forward, reward: 1.63865966366
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.6386596636595185, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.64)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: None, reward: 1.34401434462
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.344014344621906, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.34)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: forward, reward: 2.62709073357
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.627090733566117, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.63)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 3), heading: (0, -1), action: left, reward: 2.6133084489
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'left', 'left', 'left'), 'deadline': 12, 't': 8, 'action': 'left', 'reward': 2.6133084488962472, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'left', 'left', 'left')
Agent followed the waypoint left. (rewarded 2.61)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 282
\-------------------------
Environment.reset(): Trial set up with start = (1, 6), destination = (7, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0596; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: left, reward: 1.06381321619
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.0638132161897318, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.06)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 5), heading: (1, 0), action: right, reward: 1.48599827615
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.4859982761520203, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 1.49)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: left, reward: 1.54726150934
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 18, 't': 2, 'action': 'left', 'reward': 1.547261509337907, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.55)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 1.47946745324
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.4794674532375507, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.48)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (0, -1), action: None, reward: 1.74734381681
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.7473438168091238, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.75)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: left, reward: 2.07739995243
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.0773999524343036, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.08)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.11822177944
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.1182217794404665, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.12)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.70541317865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 2.7054131786547186, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.71)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.03785135663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.0378513566266594, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 1.04)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.70202594761
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 2.70202594761251, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.70)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.82306480872
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.8230648087212504, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.82)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: -5.55681495403
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 9, 't': 11, 'action': None, 'reward': -5.55681495403051, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.56)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: -4.94678544349
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 1, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 8, 't': 12, 'action': None, 'reward': -4.946785443485405, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent idled at a green light with no oncoming traffic. (rewarded -4.95)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 1.21171434592
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', None), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 1.2117143459164714, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', None)
Agent followed the waypoint forward. (rewarded 1.21)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 4), heading: (-1, 0), action: forward, reward: 2.40576148617
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 2.4057614861665577, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.41)
25% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 283
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (4, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0590; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 1.55527802346
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 1.5552780234631463, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 1.56)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: forward, reward: 1.12891273931
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': 'forward', 'reward': 1.1289127393149478, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.13)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 2.05561965116
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.055619651160641, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.06)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 2.16952378593
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.1695237859321734, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.17)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 2.69155023182
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.691550231822168, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.69)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (4, 5), heading: (-1, 0), action: None, reward: 2.04504189969
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 15, 't': 5, 'action': None, 'reward': 2.0450418996921016, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.05)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: right, reward: 1.6634958162
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.663495816199649, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.66)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 4), heading: (0, -1), action: forward, reward: -40.2491509521
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 4, 'light': 'red', 'state': ('forward', 'red', 'forward', 'forward', 'right'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': -40.24915095211569, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'forward', 'right')
Agent attempted driving forward through a red light with traffic and cause a major accident. (rewarded -40.25)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 4), heading: (1, 0), action: right, reward: 1.07106322953
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'right'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.0710632295261848, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'right')
Agent drove right instead of forward. (rewarded 1.07)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: left, reward: 1.5942109102
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 11, 't': 9, 'action': 'left', 'reward': 1.5942109102047541, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.59)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 2.33490611482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 2.3349061148231387, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.33)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 2.46004210661
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', 'left'), 'deadline': 9, 't': 11, 'action': None, 'reward': 2.460042106612776, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.46)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 1.25606690089
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 1.256066900889065, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.26)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 3), heading: (-1, 0), action: left, reward: 0.882601067174
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, 'left'), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 0.8826010671743756, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, 'left')
Agent followed the waypoint left. (rewarded 0.88)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 284
\-------------------------
Environment.reset(): Trial set up with start = (3, 5), destination = (8, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0584; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: right, reward: 0.126607487504
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'forward'), 'deadline': 20, 't': 0, 'action': 'right', 'reward': 0.1266074875038996, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'forward')
Agent drove right instead of left. (rewarded 0.13)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 5), heading: (1, 0), action: None, reward: 1.44207717422
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.442077174223845, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.44)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (0, 1), action: right, reward: 1.80660544034
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.8066054403421563, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.81)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: left, reward: 1.35185259343
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 17, 't': 3, 'action': 'left', 'reward': 1.3518525934344554, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent drove left instead of right. (rewarded 1.35)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: None, reward: 2.42414033006
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.424140330061407, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.42)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: 2.63112660249
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 2.6311266024875932, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.63)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: None, reward: 2.55119243542
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.5511924354177475, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.55)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 2.35197517917
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 2.3519751791703873, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 2.35)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: right, reward: -0.101855987226
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'right'), 'deadline': 12, 't': 8, 'action': 'right', 'reward': -0.10185598722617539, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'right')
Agent drove right instead of forward. (rewarded -0.10)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: 1.55712920177
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.5571292017672393, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.56)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (0, 1), action: None, reward: 1.32404354961
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.3240435496147531, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.32)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (7, 2), heading: (0, 1), action: forward, reward: 0.860456498779
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', None), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.8604564987791519, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', None)
Agent drove forward instead of left. (rewarded 0.86)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: right, reward: 1.07200211976
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 8, 't': 12, 'action': 'right', 'reward': 1.0720021197631913, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent drove right instead of left. (rewarded 1.07)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (6, 7), heading: (0, -1), action: right, reward: 2.08041082051
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 7, 't': 13, 'action': 'right', 'reward': 2.0804108205105907, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 2.08)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: right, reward: 2.23703280192
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 6, 't': 14, 'action': 'right', 'reward': 2.2370328019173864, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.24)
25% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.81494774004
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 5, 't': 15, 'action': None, 'reward': 1.8149477400350396, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.81)
20% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (7, 7), heading: (1, 0), action: None, reward: 1.57656631239
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 4, 't': 16, 'action': None, 'reward': 1.5765663123894622, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.58)
15% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: left, reward: 0.0934631797289
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'left'), 'deadline': 3, 't': 17, 'action': 'left', 'reward': 0.09346317972886242, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'left')
Agent drove left instead of forward. (rewarded 0.09)
10% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 0.874443269294
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 2, 't': 18, 'action': 'right', 'reward': 0.8744432692944266, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.87)
5% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 285
\-------------------------
Environment.reset(): Trial set up with start = (2, 2), destination = (1, 5), deadline = 20
Simulating trial. . .
epsilon = 0.0578; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: forward, reward: 2.73801230142
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 2.738012301423952, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', None)
Agent followed the waypoint forward. (rewarded 2.74)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 7), heading: (0, -1), action: right, reward: 1.6674898117
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.6674898117023427, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.67)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: forward, reward: 2.43420240045
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 18, 't': 2, 'action': 'forward', 'reward': 2.434202400453903, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 2.43)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 1.53147494336
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.5314749433570143, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.53)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 6), heading: (0, -1), action: None, reward: 2.37351147357
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.373511473565679, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 2.37)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: left, reward: 1.87348065549
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.87348065549222, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove left instead of forward. (rewarded 1.87)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: None, reward: 1.7861261165
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.7861261165007734, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.79)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 1.81276463733
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'forward'), 'deadline': 13, 't': 7, 'action': 'right', 'reward': 1.8127646373310404, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'forward')
Agent followed the waypoint right. (rewarded 1.81)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: right, reward: 2.82096917057
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 2.820969170570101, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.82)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 286
\-------------------------
Environment.reset(): Trial set up with start = (1, 3), destination = (4, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0573; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.05214913649
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'right'), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.052149136485659, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'right')
Agent properly idled at a red light. (rewarded 1.05)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.54936534573
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.549365345734027, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.55)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.60574954868
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.6057495486824722, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.61)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 2.02526179938
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.0252617993780087, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.03)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 2.09739900085
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 2.097399000852183, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.10)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 1.91463300498
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.9146330049839295, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.91)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: forward, reward: 1.12150363332
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'left'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.1215036333189758, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'left')
Agent followed the waypoint forward. (rewarded 1.12)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 2.77865785221
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'left', None), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.778657852211964, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.78)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 2.0017118216
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.0017118215970253, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.00)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (4, 3), heading: (1, 0), action: None, reward: 2.04240612642
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.0424061264235305, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.04)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 4), heading: (0, 1), action: right, reward: 0.868326196922
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 0.8683261969218583, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.87)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 5), heading: (0, 1), action: forward, reward: 1.63500343137
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.6350034313738653, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.64)
52% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 287
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (4, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0567; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 2.20818534268
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.208185342676251, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.21)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 1.3752519593
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', 'forward'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.3752519593014465, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.38)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: left, reward: 2.93492089917
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 2.934920899168877, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.93)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 4), heading: (-1, 0), action: None, reward: 1.44842023047
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.4484202304666032, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.45)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (0, -1), action: right, reward: 1.73468108446
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.734681084462277, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.73)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 2), heading: (0, -1), action: forward, reward: 2.64039491855
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'right'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 2.6403949185453035, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'right')
Agent followed the waypoint forward. (rewarded 2.64)
76% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 288
\-------------------------
Environment.reset(): Trial set up with start = (8, 5), destination = (5, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0561; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: right, reward: 1.17546586584
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'right'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.1754658658365633, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'right')
Agent followed the waypoint right. (rewarded 1.18)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: 2.11246675994
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 2.112466759943252, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.11)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 1.39402305
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.394023050003891, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.39)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 1.58429147979
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.5842914797886138, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.58)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 1.72067876143
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.7206787614309171, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.72)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: None, reward: 1.27271652725
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 1.272716527246504, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.27)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 1.28993237752
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.289932377519105, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.29)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 1.63521449663
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'right'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.6352144966255335, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'right')
Agent properly idled at a red light. (rewarded 1.64)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 2.62552868392
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', None), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.625528683916369, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.63)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: None, reward: 1.79088521823
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.7908852182262642, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.79)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: forward, reward: 0.998479729758
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': 'forward', 'reward': 0.9984797297582029, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.00)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 289
\-------------------------
Environment.reset(): Trial set up with start = (5, 6), destination = (1, 3), deadline = 35
Simulating trial. . .
epsilon = 0.0556; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: left, reward: -20.812129314
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': 'forward'}, 'violation': 3, 'light': 'green', 'state': ('forward', 'green', 'forward', 'right', 'forward'), 'deadline': 35, 't': 0, 'action': 'left', 'reward': -20.812129314027523, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'right', 'forward')
Agent attempted driving left through traffic and cause a minor accident. (rewarded -20.81)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 6), heading: (1, 0), action: forward, reward: 1.76804269125
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 34, 't': 1, 'action': 'forward', 'reward': 1.768042691245312, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.77)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: forward, reward: 2.31520437606
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 33, 't': 2, 'action': 'forward', 'reward': 2.315204376062439, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 2.32)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (1, 0), action: None, reward: 2.71547369921
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 32, 't': 3, 'action': None, 'reward': 2.7154736992059996, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.72)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: forward, reward: 2.4297741373
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 31, 't': 4, 'action': 'forward', 'reward': 2.4297741373012567, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 2.43)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 1.66263311757
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 30, 't': 5, 'action': 'forward', 'reward': 1.66263311756638, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.66)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 2.909730372
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', None, None), 'deadline': 29, 't': 6, 'action': 'right', 'reward': 2.909730371999876, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', None, None)
Agent followed the waypoint right. (rewarded 2.91)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 1.43438664796
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 7, 'action': None, 'reward': 1.4343866479560579, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.43)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: None, reward: 2.65389318598
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 27, 't': 8, 'action': None, 'reward': 2.6538931859829358, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 2.65)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (0, 1), action: forward, reward: 1.82102141619
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 26, 't': 9, 'action': 'forward', 'reward': 1.8210214161916956, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.82)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: forward, reward: 1.69233368645
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 25, 't': 10, 'action': 'forward', 'reward': 1.6923336864466114, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 1.69)
69% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 290
\-------------------------
Environment.reset(): Trial set up with start = (2, 3), destination = (7, 7), deadline = 25
Simulating trial. . .
epsilon = 0.0550; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 2), heading: (0, -1), action: right, reward: 0.997225046145
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 0.9972250461450034, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', 'forward')
Agent drove right instead of forward. (rewarded 1.00)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: forward, reward: 1.00623243501
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 24, 't': 1, 'action': 'forward', 'reward': 1.0062324350101095, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent drove forward instead of left. (rewarded 1.01)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: left, reward: 2.44272790081
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 2.4427279008087144, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 2.44)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 1.67905080428
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.6790508042764052, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.68)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 7), heading: (-1, 0), action: None, reward: 2.3154287901
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.3154287900953356, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.32)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: forward, reward: 1.76871131
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.7687113100025948, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.77)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 7), heading: (-1, 0), action: forward, reward: 1.03485309433
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.0348530943292804, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.03)
72% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 291
\-------------------------
Environment.reset(): Trial set up with start = (6, 2), destination = (1, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0545; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0545; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0545; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.78673584617
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', None), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.7867358461730043, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', None)
Agent properly idled at a red light. (rewarded 2.79)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.6984629482
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.69846294820292, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.70)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.12200245336
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': None, 'reward': 1.1220024533564266, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.12)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 1.56922945335
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.5692294533523663, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.57)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (1, 0), action: None, reward: 2.59425206799
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.5942520679868037, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.59)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: right, reward: 0.905939777289
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.9059397772885018, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', 'forward')
Agent drove right instead of forward. (rewarded 0.91)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: None, reward: 2.53257762587
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.5325776258662267, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.53)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: left, reward: 2.07751496951
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 2.077514969505253, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.08)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: -9.0047825815
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 2, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': -9.004782581497363, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent attempted driving forward through a red light. (rewarded -9.00)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: None, reward: 1.62812796537
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.6281279653742173, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.63)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: forward, reward: 2.34680917284
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'forward'), 'deadline': 10, 't': 10, 'action': 'forward', 'reward': 2.3468091728369633, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'forward')
Agent followed the waypoint forward. (rewarded 2.35)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 0.897045486757
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 11, 'action': None, 'reward': 0.8970454867571727, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 0.90)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 3), heading: (1, 0), action: None, reward: 2.0648126452
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.064812645195369, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.06)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: forward, reward: 2.55247644991
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 7, 't': 13, 'action': 'forward', 'reward': 2.5524764499061217, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.55)
30% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 292
\-------------------------
Environment.reset(): Trial set up with start = (6, 5), destination = (1, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0539; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (0, 1), action: None, reward: 2.53503777856
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'right', 'left', 'right'), 'deadline': 20, 't': 0, 'action': None, 'reward': 2.5350377785640674, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'right', 'left', 'right')
Agent properly idled at a red light. (rewarded 2.54)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: right, reward: 1.29026234509
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'right'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 1.290262345092116, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'right')
Agent drove right instead of left. (rewarded 1.29)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (5, 4), heading: (0, -1), action: right, reward: 0.987954346482
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 0.987954346481604, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 0.99)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 4), heading: (1, 0), action: right, reward: 2.135621732
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 17, 't': 3, 'action': 'right', 'reward': 2.1356217319981634, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.14)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 4), heading: (1, 0), action: forward, reward: 0.943348105336
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', None), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 0.9433481053357533, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', None)
Agent followed the waypoint forward. (rewarded 0.94)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 4), heading: (1, 0), action: forward, reward: 2.45101773621
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 2.451017736210532, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.45)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: right, reward: 1.86843187848
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'forward', None), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.8684318784799736, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'forward', None)
Agent drove right instead of forward. (rewarded 1.87)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: left, reward: 1.2995896966
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 1.2995896965983256, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.30)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 1.75715323717
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.7571532371735032, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.76)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 5), heading: (1, 0), action: None, reward: 1.39275973151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.3927597315084268, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.39)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: right, reward: 1.51160099766
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'forward', 'left'), 'deadline': 10, 't': 10, 'action': 'right', 'reward': 1.5116009976589977, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.51)
45% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 293
\-------------------------
Environment.reset(): Trial set up with start = (6, 7), destination = (1, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0534; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: 2.51596118925
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 25, 't': 0, 'action': 'left', 'reward': 2.51596118924901, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.52)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 2.17372002211
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'right', None), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.1737200221078217, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'right', None)
Agent properly idled at a red light. (rewarded 2.17)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 1.29264271931
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.2926427193062366, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.29)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 1.70951961624
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.7095196162393123, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.71)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: None, reward: 0.992049637466
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 0.9920496374655694, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 0.99)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (1, 0), action: left, reward: 1.3073624962
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 1.3073624962040427, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 1.31)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: forward, reward: 2.18436338458
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.184363384579374, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.18)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 2.01271851586
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.0127185158624203, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.01)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (1, 0), action: None, reward: 1.92486116501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.92486116500918, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 1.92)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 1.80505002042
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.8050500204184745, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.81)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: right, reward: 2.30349494811
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 2.30349494811234, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 2.30)
56% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 294
\-------------------------
Environment.reset(): Trial set up with start = (4, 5), destination = (1, 2), deadline = 30
Simulating trial. . .
epsilon = 0.0529; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: left, reward: 2.38830793533
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 30, 't': 0, 'action': 'left', 'reward': 2.388307935331014, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.39)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 2.79131676843
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 29, 't': 1, 'action': None, 'reward': 2.791316768429748, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 2.79)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 1.58683291655
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.5868329165461452, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.59)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 1.7106362041
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.7106362041001257, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.71)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 5), heading: (-1, 0), action: None, reward: 1.41968116832
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.4196811683213124, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.42)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 5), heading: (-1, 0), action: forward, reward: 2.0212434529
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, None), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 2.021243452902776, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, None)
Agent followed the waypoint forward. (rewarded 2.02)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: 2.80337479929
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 24, 't': 6, 'action': 'forward', 'reward': 2.803374799291639, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 2.80)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.36711106958
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 23, 't': 7, 'action': None, 'reward': 2.3671110695771134, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 2.37)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: forward, reward: -10.1751769114
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 2, 'light': 'red', 'state': ('left', 'red', 'left', 'right', None), 'deadline': 22, 't': 8, 'action': 'forward', 'reward': -10.17517691144706, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'right', None)
Agent attempted driving forward through a red light. (rewarded -10.18)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 5), heading: (-1, 0), action: None, reward: 2.83968071248
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 21, 't': 9, 'action': None, 'reward': 2.839680712476535, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.84)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 6), heading: (0, 1), action: left, reward: 2.64307302295
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 10, 'action': 'left', 'reward': 2.6430730229503174, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.64)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: forward, reward: 2.46419306841
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 19, 't': 11, 'action': 'forward', 'reward': 2.464193068407652, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent followed the waypoint forward. (rewarded 2.46)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: 0.542928776263
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 18, 't': 12, 'action': 'right', 'reward': 0.542928776263443, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent drove right instead of forward. (rewarded 0.54)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 0.428368783391
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', 'left'), 'deadline': 17, 't': 13, 'action': None, 'reward': 0.4283687833909504, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 0.43)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: None, reward: 1.07383939291
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', 'left'), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.0738393929104668, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.07)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: left, reward: 2.21023346288
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 15, 't': 15, 'action': 'left', 'reward': 2.2102334628791116, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 2.21)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: -5.97347168958
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 1, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', None), 'deadline': 14, 't': 16, 'action': None, 'reward': -5.973471689575227, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', None)
Agent idled at a green light with no oncoming traffic. (rewarded -5.97)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 0.754300325729
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', None), 'deadline': 13, 't': 17, 'action': None, 'reward': 0.7543003257290755, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 0.75)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (8, 2), heading: (0, 1), action: None, reward: 2.28332808151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 12, 't': 18, 'action': None, 'reward': 2.2833280815087758, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.28)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 0.466279581724
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'right'), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 0.4662795817243418, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'right')
Agent drove right instead of left. (rewarded 0.47)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 0.954457872117
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 0.9544578721174479, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 0.95)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (7, 6), heading: (0, -1), action: forward, reward: 0.273651004076
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 9, 't': 21, 'action': 'forward', 'reward': 0.27365100407626175, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent drove forward instead of right. (rewarded 0.27)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: right, reward: 2.11558178579
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 2.115581785789249, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.12)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 1.71150605674
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 7, 't': 23, 'action': None, 'reward': 1.711506056738137, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.71)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 0.504898713576
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'right'), 'deadline': 6, 't': 24, 'action': None, 'reward': 0.5048987135756495, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 0.50)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 1.48316507997
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'right'), 'deadline': 5, 't': 25, 'action': 'forward', 'reward': 1.483165079969103, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'right')
Agent followed the waypoint forward. (rewarded 1.48)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.29100414091
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 4, 't': 26, 'action': None, 'reward': 1.2910041409053008, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 1.29)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 2.15350120019
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', None), 'deadline': 3, 't': 27, 'action': None, 'reward': 2.1535012001860308, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.15)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (1, 7), heading: (0, 1), action: right, reward: 1.89335952936
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 28, 'action': 'right', 'reward': 1.8933595293614502, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.89)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (8, 7), heading: (-1, 0), action: right, reward: -0.806761626694
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', None, 'forward'), 'deadline': 1, 't': 29, 'action': 'right', 'reward': -0.8067616266942099, 'waypoint': 'forward'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('forward', 'green', 'right', None, 'forward')
Agent drove right instead of forward. (rewarded -0.81)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Training trial 295
\-------------------------
Environment.reset(): Trial set up with start = (5, 3), destination = (8, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0523; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0523; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (5, 3), heading: (0, -1), action: None, reward: 1.72693293197
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', 'forward', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 1.72693293196687, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', 'forward', None)
Agent properly idled at a red light. (rewarded 1.73)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: right, reward: 1.48087679376
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.480876793757457, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', None, 'left')
Agent followed the waypoint right. (rewarded 1.48)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: 1.54089122609
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 1.5408912260921865, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent followed the waypoint forward. (rewarded 1.54)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 4), heading: (0, 1), action: right, reward: 0.704563547069
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'left', None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 0.7045635470690355, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'left', None)
Agent drove right instead of forward. (rewarded 0.70)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: forward, reward: 0.533860379214
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'right', None), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 0.5338603792140796, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'right', None)
Agent drove forward instead of left. (rewarded 0.53)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 2.89725353608
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', 'forward', 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.8972535360846647, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.90)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.90212810522
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.9021281052226864, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.90)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 2.44201206922
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 2.4420120692224296, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.44)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (7, 5), heading: (0, 1), action: None, reward: 1.54009328067
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 1.5400932806720855, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.54)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: left, reward: 2.36270671764
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'left', 'reward': 2.362706717637259, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 2.36)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 296
\-------------------------
Environment.reset(): Trial set up with start = (8, 6), destination = (2, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0518; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 6), heading: (-1, 0), action: forward, reward: 1.46098141658
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'right', 'right'), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.4609814165750055, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'right', 'right')
Agent drove forward instead of right. (rewarded 1.46)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (0, -1), action: right, reward: 2.53392502623
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 19, 't': 1, 'action': 'right', 'reward': 2.533925026231217, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 2.53)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: right, reward: 1.18435184793
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'forward'), 'deadline': 18, 't': 2, 'action': 'right', 'reward': 1.1843518479323345, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'forward')
Agent followed the waypoint right. (rewarded 1.18)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 2.39079114266
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'forward', None), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.3907911426637214, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.39)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (8, 5), heading: (1, 0), action: None, reward: 1.90585618311
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 16, 't': 4, 'action': None, 'reward': 1.9058561831128553, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.91)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: left, reward: 1.11764281817
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'left', 'left'), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 1.117642818171689, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'left', 'left')
Agent drove left instead of forward. (rewarded 1.12)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: right, reward: 1.47573102237
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', 'left'), 'deadline': 14, 't': 6, 'action': 'right', 'reward': 1.4757310223661462, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', 'left')
Agent followed the waypoint right. (rewarded 1.48)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.93008140419
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.9300814041937733, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.93)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 2.41074638872
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 12, 't': 8, 'action': None, 'reward': 2.4107463887237293, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.41)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.507708309
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.5077083090045875, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.51)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (1, 0), action: None, reward: 1.74163352366
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.7416335236563028, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.74)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 4), heading: (1, 0), action: forward, reward: 2.66808993677
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 2.6680899367705964, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 2.67)
40% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 297
\-------------------------
Environment.reset(): Trial set up with start = (1, 2), destination = (3, 4), deadline = 20
Simulating trial. . .
epsilon = 0.0513; alpha = 0.5000
Simulating trial. . .
epsilon = 0.0513; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: left, reward: 2.66110874287
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 2.661108742865743, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'left', 'left')
Agent followed the waypoint left. (rewarded 2.66)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 2.50042821472
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 2.500428214724092, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.50)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 2.3274419356
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.3274419356034683, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.33)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (0, 1), action: None, reward: 2.34245977185
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.342459771854773, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.34)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: left, reward: 1.24373297164
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 16, 't': 4, 'action': 'left', 'reward': 1.2437329716420953, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.24)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: right, reward: 0.986314010544
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, 'forward'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 0.9863140105442878, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, 'forward')
Agent drove right instead of forward. (rewarded 0.99)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 4), heading: (1, 0), action: left, reward: 0.998029472752
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', None, None), 'deadline': 14, 't': 6, 'action': 'left', 'reward': 0.9980294727519878, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', None, None)
Agent followed the waypoint left. (rewarded 1.00)
65% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 298
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (4, 2), deadline = 20
Simulating trial. . .
epsilon = 0.0508; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: forward, reward: 1.72043351176
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 0, 'action': 'forward', 'reward': 1.7204335117639513, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.72)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 1.12759447263
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.1275944726268283, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.13)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 2.10419151535
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.104191515350915, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.10)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 2.73844682433
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 2.7384468243341993, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.74)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (1, 0), action: None, reward: 2.90536772484
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.905367724843993, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.91)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: forward, reward: 1.87398495041
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'forward', 'reward': 1.8739849504100772, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.87)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 2), heading: (1, 0), action: forward, reward: 1.33318768019
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 14, 't': 6, 'action': 'forward', 'reward': 1.3331876801929015, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.33)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (3, 7), heading: (0, -1), action: left, reward: 0.160794063383
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'left'), 'deadline': 13, 't': 7, 'action': 'left', 'reward': 0.16079406338270008, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'left')
Agent drove left instead of forward. (rewarded 0.16)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: right, reward: 1.15494763034
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 12, 't': 8, 'action': 'right', 'reward': 1.1549476303448165, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.15)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: right, reward: 2.28006439501
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': 'right', 'reward': 2.28006439501486, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.28)
50% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 299
\-------------------------
Environment.reset(): Trial set up with start = (1, 4), destination = (5, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0503; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (1, 5), heading: (0, 1), action: right, reward: 1.9511367452
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'forward'), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.9511367451951296, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'forward')
Agent followed the waypoint right. (rewarded 1.95)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: right, reward: 2.74415974018
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'forward', None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 2.7441597401845104, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'forward', None)
Agent followed the waypoint right. (rewarded 2.74)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 5), heading: (-1, 0), action: forward, reward: 1.65637713292
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 23, 't': 2, 'action': 'forward', 'reward': 1.656377132918733, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent followed the waypoint forward. (rewarded 1.66)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: forward, reward: 1.82038587882
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 1.8203858788247267, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 1.82)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (6, 5), heading: (-1, 0), action: None, reward: 2.46797666919
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.467976669193803, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.47)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 5), heading: (-1, 0), action: forward, reward: 1.76163444817
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.7616344481679935, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.76)
76% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Training trial 300
\-------------------------
Environment.reset(): Trial set up with start = (6, 6), destination = (2, 3), deadline = 35
Simulating trial. . .
epsilon = 0.0498; alpha = 0.5000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (6, 5), heading: (0, -1), action: right, reward: 0.139544038321
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 35, 't': 0, 'action': 'right', 'reward': 0.13954403832116058, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent drove right instead of left. (rewarded 0.14)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 5), heading: (1, 0), action: right, reward: 2.34764187603
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 34, 't': 1, 'action': 'right', 'reward': 2.3476418760323527, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.35)
94% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: right, reward: 1.98396579899
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'forward'), 'deadline': 33, 't': 2, 'action': 'right', 'reward': 1.9839657989914126, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'forward')
Agent drove right instead of forward. (rewarded 1.98)
91% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 1.66036093615
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', 'left'), 'deadline': 32, 't': 3, 'action': None, 'reward': 1.66036093614969, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.66)
89% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 1.55504407365
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'left', None), 'deadline': 31, 't': 4, 'action': None, 'reward': 1.555044073654728, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.56)
86% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 2.85647176042
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', None), 'deadline': 30, 't': 5, 'action': None, 'reward': 2.856471760421277, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 2.86)
83% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (7, 6), heading: (0, 1), action: None, reward: 2.87856009395
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 29, 't': 6, 'action': None, 'reward': 2.8785600939485287, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.88)
80% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: left, reward: 1.61700089024
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'left'), 'deadline': 28, 't': 7, 'action': 'left', 'reward': 1.6170008902374378, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'left')
Agent followed the waypoint left. (rewarded 1.62)
77% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 2.46275924649
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 27, 't': 8, 'action': None, 'reward': 2.4627592464886137, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.46)
74% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (8, 6), heading: (1, 0), action: None, reward: 1.57206837119
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 26, 't': 9, 'action': None, 'reward': 1.572068371190741, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.57)
71% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: forward, reward: 1.0112531728
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', None, 'left'), 'deadline': 25, 't': 10, 'action': 'forward', 'reward': 1.0112531727973946, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', None, 'left')
Agent followed the waypoint forward. (rewarded 1.01)
69% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 6), heading: (1, 0), action: None, reward: 1.4308362025
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', 'left'), 'deadline': 24, 't': 11, 'action': None, 'reward': 1.430836202504158, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', 'left')
Agent idled at a green light with oncoming traffic. (rewarded 1.43)
66% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (2, 6), heading: (1, 0), action: forward, reward: 2.2167759384
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 23, 't': 12, 'action': 'forward', 'reward': 2.2167759383961085, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.22)
63% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (2, 7), heading: (0, 1), action: right, reward: 2.11457420308
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 22, 't': 13, 'action': 'right', 'reward': 2.1145742030819914, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 2.11)
60% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: forward, reward: 1.93115671761
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', None, None), 'deadline': 21, 't': 14, 'action': 'forward', 'reward': 1.931156717606009, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', None, None)
Agent followed the waypoint forward. (rewarded 1.93)
57% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (2, 3), heading: (0, 1), action: forward, reward: 1.44423470658
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 20, 't': 15, 'action': 'forward', 'reward': 1.4442347065825305, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.44)
54% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 1
\-------------------------
Environment.reset(): Trial set up with start = (2, 7), destination = (6, 3), deadline = 30
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 2), heading: (0, 1), action: right, reward: 1.31817574999
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 30, 't': 0, 'action': 'right', 'reward': 1.318175749988308, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.32)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: right, reward: 1.37491012475
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'left', None), 'deadline': 29, 't': 1, 'action': 'right', 'reward': 1.374910124745394, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'left', None)
Agent followed the waypoint right. (rewarded 1.37)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 2.5465259824
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 2.5465259823960054, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.55)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 2.21036395003
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 2.210363950028171, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.21)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 2), heading: (-1, 0), action: None, reward: 1.78933200035
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.7893320003462994, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.79)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: forward, reward: 2.37781887818
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 25, 't': 5, 'action': 'forward', 'reward': 2.377818878177915, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.38)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 2.66829286165
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 6, 'action': None, 'reward': 2.668292861650813, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.67)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 2.70774859639
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 23, 't': 7, 'action': None, 'reward': 2.707748596394057, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 2.71)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 2), heading: (-1, 0), action: None, reward: 1.75686349555
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'forward'), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.7568634955454452, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'forward')
Agent properly idled at a red light. (rewarded 1.76)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: forward, reward: 2.45895554561
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 21, 't': 9, 'action': 'forward', 'reward': 2.458955545611639, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.46)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (7, 7), heading: (0, -1), action: right, reward: 1.59009758131
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', 'left'), 'deadline': 20, 't': 10, 'action': 'right', 'reward': 1.5900975813121019, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', 'left')
Agent drove right instead of forward. (rewarded 1.59)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (6, 7), heading: (-1, 0), action: left, reward: 2.2551490379
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 19, 't': 11, 'action': 'left', 'reward': 2.255149037902534, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.26)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: left, reward: 1.65596142895
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'right', None), 'deadline': 18, 't': 12, 'action': 'left', 'reward': 1.6559614289491968, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'right', None)
Agent followed the waypoint left. (rewarded 1.66)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (6, 3), heading: (0, 1), action: forward, reward: 1.11145582891
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 1.1114558289131637, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 1.11)
53% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 2
\-------------------------
Environment.reset(): Trial set up with start = (2, 4), destination = (8, 6), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 1.56290822834
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'left', 'left', 'left'), 'deadline': 20, 't': 0, 'action': None, 'reward': 1.562908228343746, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'left', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.56)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 1.83622188527
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 19, 't': 1, 'action': None, 'reward': 1.8362218852697727, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.84)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 2.64246406119
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.6424640611856054, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.64)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 1.23618295881
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'left', 'left'), 'deadline': 17, 't': 3, 'action': None, 'reward': 1.2361829588123574, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.24)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 4), heading: (0, 1), action: None, reward: 2.14034239045
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'left'), 'deadline': 16, 't': 4, 'action': None, 'reward': 2.1403423904506815, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.14)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: right, reward: 1.40395840238
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, 'left'), 'deadline': 15, 't': 5, 'action': 'right', 'reward': 1.4039584023846912, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, 'left')
Agent followed the waypoint right. (rewarded 1.40)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.55530974706
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 14, 't': 6, 'action': None, 'reward': 1.5553097470626505, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 1.56)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.98975989918
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', None), 'deadline': 13, 't': 7, 'action': None, 'reward': 1.9897598991826293, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', None)
Agent properly idled at a red light. (rewarded 1.99)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.72873477643
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, None), 'deadline': 12, 't': 8, 'action': None, 'reward': 1.728734776425741, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 1.73)
55% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.98918651294
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 11, 't': 9, 'action': None, 'reward': 1.9891865129387665, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.99)
50% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.01031100811
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 10, 't': 10, 'action': None, 'reward': 1.0103110081118076, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.01)
45% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 0.907055385914
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', 'right'), 'deadline': 9, 't': 11, 'action': 'forward', 'reward': 0.9070553859137596, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', 'right')
Agent followed the waypoint forward. (rewarded 0.91)
40% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: None, reward: 2.57911693466
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 8, 't': 12, 'action': None, 'reward': 2.579116934659626, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.58)
35% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 5), heading: (0, 1), action: left, reward: 1.40397623487
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 7, 't': 13, 'action': 'left', 'reward': 1.4039762348743674, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.40)
30% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 6), heading: (0, 1), action: forward, reward: 1.9057515118
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'left', 'left', None), 'deadline': 6, 't': 14, 'action': 'forward', 'reward': 1.905751511795554, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'left', 'left', None)
Agent followed the waypoint forward. (rewarded 1.91)
25% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 3
\-------------------------
Environment.reset(): Trial set up with start = (8, 3), destination = (3, 5), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: None, reward: 2.41379416751
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, 'forward', None), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.41379416751188, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, 'forward', None)
Agent properly idled at a red light. (rewarded 2.41)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: right, reward: 1.02837172372
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.0283717237188736, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 1.03)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.90757792828
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.9075779282828582, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.91)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 3), heading: (1, 0), action: None, reward: 1.16234917712
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.1623491771189245, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.16)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: forward, reward: 1.58410051531
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 21, 't': 4, 'action': 'forward', 'reward': 1.5841005153087624, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.58)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 2.74370904232
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.743709042324222, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', None)
Agent properly idled at a red light. (rewarded 2.74)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.85821420357
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.8582142035723646, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.86)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 1.69737436752
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.6973743675215518, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.70)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 3), heading: (1, 0), action: None, reward: 2.47302093403
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 2.4730209340287823, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 2.47)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 3), heading: (1, 0), action: forward, reward: 1.64911853017
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 1.6491185301657554, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.65)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (3, 4), heading: (0, 1), action: right, reward: 1.08168377317
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 15, 't': 10, 'action': 'right', 'reward': 1.0816837731684075, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.08)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (3, 5), heading: (0, 1), action: forward, reward: 1.32893497181
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'right'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.3289349718050443, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'right')
Agent followed the waypoint forward. (rewarded 1.33)
52% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 4
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (8, 2), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: right, reward: 2.1548266676
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.154826667601397, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 2.15)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 1.31455128931
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.3145512893129776, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.31)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: None, reward: 1.77000896121
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'forward'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.7700089612140097, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.77)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: 2.7806602763
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 22, 't': 3, 'action': 'forward', 'reward': 2.7806602763027457, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 2.78)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (1, 5), heading: (0, -1), action: right, reward: 1.55408123477
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'right', 'left'), 'deadline': 21, 't': 4, 'action': 'right', 'reward': 1.5540812347733866, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'right', 'left')
Agent drove right instead of forward. (rewarded 1.55)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: left, reward: 0.941524642169
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 20, 't': 5, 'action': 'left', 'reward': 0.9415246421686909, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 0.94)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (8, 5), heading: (-1, 0), action: None, reward: 1.29301931262
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 19, 't': 6, 'action': None, 'reward': 1.2930193126166531, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 1.29)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: right, reward: 0.662977596845
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 18, 't': 7, 'action': 'right', 'reward': 0.6629775968445575, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.66)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: forward, reward: 2.67992478409
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 2.679924784090306, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.68)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 2), heading: (0, -1), action: forward, reward: 2.83729931876
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.8372993187621374, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 2.84)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 5
\-------------------------
Environment.reset(): Trial set up with start = (3, 6), destination = (8, 4), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: None, reward: 2.91775656459
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'left'), 'deadline': 25, 't': 0, 'action': None, 'reward': 2.9177565645857566, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 2.92)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: right, reward: 1.41675274004
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, 'left'), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.4167527400403512, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, 'left')
Agent drove right instead of left. (rewarded 1.42)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 7), heading: (0, 1), action: right, reward: 1.20594421075
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'forward', 'right'), 'deadline': 23, 't': 2, 'action': 'right', 'reward': 1.2059442107470844, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'forward', 'right')
Agent drove right instead of left. (rewarded 1.21)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: right, reward: 2.48685095937
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'forward', None), 'deadline': 22, 't': 3, 'action': 'right', 'reward': 2.48685095937281, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'forward', None)
Agent followed the waypoint right. (rewarded 2.49)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.55501933387
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'left', None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.5550193338718534, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'left', None)
Agent properly idled at a red light. (rewarded 2.56)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (3, 7), heading: (-1, 0), action: None, reward: 2.0192910145
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', None), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.019291014502545, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', None)
Agent properly idled at a red light. (rewarded 2.02)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (3, 6), heading: (0, -1), action: right, reward: 1.22365965758
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', 'right', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.223659657582857, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', 'right', None)
Agent drove right instead of forward. (rewarded 1.22)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 6), heading: (-1, 0), action: left, reward: 1.60448287748
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', None), 'deadline': 18, 't': 7, 'action': 'left', 'reward': 1.604482877479478, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', None)
Agent followed the waypoint left. (rewarded 1.60)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: forward, reward: 1.32844318401
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 17, 't': 8, 'action': 'forward', 'reward': 1.3284431840147415, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.33)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.82371133326
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'forward', 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 1.8237113332626242, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.82)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.85333427292
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 1.8533342729228957, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.85)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 1.43677022003
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 14, 't': 11, 'action': None, 'reward': 1.4367702200258086, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.44)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (1, 6), heading: (-1, 0), action: None, reward: 2.66837145598
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 13, 't': 12, 'action': None, 'reward': 2.6683714559757683, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 2.67)
48% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (8, 6), heading: (-1, 0), action: forward, reward: 0.833852307959
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 12, 't': 13, 'action': 'forward', 'reward': 0.8338523079588736, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.83)
44% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: right, reward: 1.89060041657
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'right', 'left'), 'deadline': 11, 't': 14, 'action': 'right', 'reward': 1.890600416572189, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'right', 'left')
Agent followed the waypoint right. (rewarded 1.89)
40% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 0.881709976017
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, None), 'deadline': 10, 't': 15, 'action': None, 'reward': 0.8817099760166065, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, None)
Agent properly idled at a red light. (rewarded 0.88)
36% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (8, 5), heading: (0, -1), action: None, reward: 1.79797177113
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 9, 't': 16, 'action': None, 'reward': 1.7979717711270689, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.80)
32% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 4), heading: (0, -1), action: forward, reward: 1.87635054271
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'left', 'forward'), 'deadline': 8, 't': 17, 'action': 'forward', 'reward': 1.8763505427054836, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'left', 'forward')
Agent followed the waypoint forward. (rewarded 1.88)
28% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 6
\-------------------------
Environment.reset(): Trial set up with start = (4, 4), destination = (8, 3), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: right, reward: 1.4690296686
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'right', 'right', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 1.4690296685986886, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'right', 'right', None)
Agent followed the waypoint right. (rewarded 1.47)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 1.70920832352
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', 'left', 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 1.7092083235221178, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', 'left', 'left')
Agent properly idled at a red light. (rewarded 1.71)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 1.22611207686
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'left', 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.2261120768576583, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'left', 'left')
Agent properly idled at a red light. (rewarded 1.23)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 2.50437588012
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.5043758801217066, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.50)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (3, 4), heading: (-1, 0), action: None, reward: 2.23625435151
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.2362543515064477, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.24)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 4), heading: (-1, 0), action: forward, reward: 0.968656853911
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 0.9686568539111251, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 0.97)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: forward, reward: 2.76746274754
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 2.7674627475404305, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.77)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 1.65078746124
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 7, 'action': None, 'reward': 1.6507874612361897, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.65)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 0.915356552121
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 17, 't': 8, 'action': None, 'reward': 0.9153565521211984, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 0.92)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.697189278
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, 'left'), 'deadline': 16, 't': 9, 'action': None, 'reward': 2.697189277995469, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, 'left')
Agent properly idled at a red light. (rewarded 2.70)
60% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (1, 4), heading: (-1, 0), action: None, reward: 2.45249701856
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 15, 't': 10, 'action': None, 'reward': 2.452497018562796, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.45)
56% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (8, 4), heading: (-1, 0), action: forward, reward: 1.59900830358
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 14, 't': 11, 'action': 'forward', 'reward': 1.5990083035794525, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.60)
52% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (8, 3), heading: (0, -1), action: right, reward: 2.53317048438
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'left'), 'deadline': 13, 't': 12, 'action': 'right', 'reward': 2.533170484382869, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'left')
Agent followed the waypoint right. (rewarded 2.53)
48% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 7
\-------------------------
Environment.reset(): Trial set up with start = (2, 5), destination = (5, 7), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (3, 5), heading: (1, 0), action: right, reward: 2.45533616293
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.455336162926619, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 2.46)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (3, 6), heading: (0, 1), action: right, reward: 1.47288280881
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'right', 'forward', None), 'deadline': 24, 't': 1, 'action': 'right', 'reward': 1.4728828088065768, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'right', 'forward', None)
Agent drove right instead of forward. (rewarded 1.47)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: left, reward: 1.00222095039
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, None), 'deadline': 23, 't': 2, 'action': 'left', 'reward': 1.002220950388105, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, None)
Agent followed the waypoint left. (rewarded 1.00)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 1.7266729768
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'forward', None, None), 'deadline': 22, 't': 3, 'action': None, 'reward': 1.726672976803822, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 1.73)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 6), heading: (1, 0), action: None, reward: 2.60041392667
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 21, 't': 4, 'action': None, 'reward': 2.6004139266714796, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.60)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 6), heading: (1, 0), action: forward, reward: 1.96917257858
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'forward'), 'deadline': 20, 't': 5, 'action': 'forward', 'reward': 1.9691725785819973, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'forward')
Agent followed the waypoint forward. (rewarded 1.97)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 7), heading: (0, 1), action: right, reward: 1.5453992604
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'left', None), 'deadline': 19, 't': 6, 'action': 'right', 'reward': 1.5453992604025923, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'left', None)
Agent followed the waypoint right. (rewarded 1.55)
72% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 8
\-------------------------
Environment.reset(): Trial set up with start = (2, 2), destination = (6, 6), deadline = 30
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.76521960555
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'forward', 'forward', 'left'), 'deadline': 30, 't': 0, 'action': None, 'reward': 1.7652196055533753, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'forward', 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.77)
97% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.6265822766
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'forward', 'left'), 'deadline': 29, 't': 1, 'action': None, 'reward': 1.6265822765997784, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'forward', 'left')
Agent properly idled at a red light. (rewarded 1.63)
93% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.67729397659
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 28, 't': 2, 'action': None, 'reward': 1.677293976590291, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.68)
90% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.12552103789
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 27, 't': 3, 'action': None, 'reward': 1.1255210378856688, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.13)
87% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (2, 2), heading: (1, 0), action: None, reward: 1.8787282909
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'left'), 'deadline': 26, 't': 4, 'action': None, 'reward': 1.8787282908974021, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.88)
83% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: left, reward: 2.01139118935
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', None, 'left'), 'deadline': 25, 't': 5, 'action': 'left', 'reward': 2.0113911893462015, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', None, 'left')
Agent followed the waypoint left. (rewarded 2.01)
80% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.10704053517
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 24, 't': 6, 'action': None, 'reward': 1.1070405351662511, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.11)
77% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.15454892031
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 23, 't': 7, 'action': None, 'reward': 1.154548920307874, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.15)
73% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (2, 7), heading: (0, -1), action: None, reward: 1.39272712342
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 22, 't': 8, 'action': None, 'reward': 1.3927271234213041, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 1.39)
70% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act() [POST]: location: (3, 7), heading: (1, 0), action: right, reward: 0.938963356168
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, 'forward'), 'deadline': 21, 't': 9, 'action': 'right', 'reward': 0.9389633561678719, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, 'forward')
Agent drove right instead of left. (rewarded 0.94)
67% of time remaining to reach destination.
/-------------------
| Step 10 Results
\-------------------
Environment.step(): t = 10
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: forward, reward: 1.52874308356
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'forward', None), 'deadline': 20, 't': 10, 'action': 'forward', 'reward': 1.5287430835603568, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'forward', None)
Agent followed the waypoint forward. (rewarded 1.53)
63% of time remaining to reach destination.
/-------------------
| Step 11 Results
\-------------------
Environment.step(): t = 11
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 1.23916765257
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 19, 't': 11, 'action': None, 'reward': 1.2391676525703914, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.24)
60% of time remaining to reach destination.
/-------------------
| Step 12 Results
\-------------------
Environment.step(): t = 12
Environment.act() [POST]: location: (4, 7), heading: (1, 0), action: None, reward: 1.13923407921
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, None), 'deadline': 18, 't': 12, 'action': None, 'reward': 1.1392340792089513, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.14)
57% of time remaining to reach destination.
/-------------------
| Step 13 Results
\-------------------
Environment.step(): t = 13
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: forward, reward: 2.49237678141
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 17, 't': 13, 'action': 'forward', 'reward': 2.4923767814105946, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.49)
53% of time remaining to reach destination.
/-------------------
| Step 14 Results
\-------------------
Environment.step(): t = 14
Environment.act() [POST]: location: (5, 7), heading: (1, 0), action: None, reward: 1.10334379009
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', 'left', 'forward'), 'deadline': 16, 't': 14, 'action': None, 'reward': 1.1033437900885796, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', 'left', 'forward')
Agent properly idled at a red light. (rewarded 1.10)
50% of time remaining to reach destination.
/-------------------
| Step 15 Results
\-------------------
Environment.step(): t = 15
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: forward, reward: 1.80533436675
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, 'right', 'forward'), 'deadline': 15, 't': 15, 'action': 'forward', 'reward': 1.8053343667544723, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, 'right', 'forward')
Agent followed the waypoint forward. (rewarded 1.81)
47% of time remaining to reach destination.
/-------------------
| Step 16 Results
\-------------------
Environment.step(): t = 16
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 2.42041741431
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 14, 't': 16, 'action': None, 'reward': 2.420417414313631, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 2.42)
43% of time remaining to reach destination.
/-------------------
| Step 17 Results
\-------------------
Environment.step(): t = 17
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.547610998
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 13, 't': 17, 'action': None, 'reward': 1.5476109980030193, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.55)
40% of time remaining to reach destination.
/-------------------
| Step 18 Results
\-------------------
Environment.step(): t = 18
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 0.851854213159
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', 'left', None, None), 'deadline': 12, 't': 18, 'action': None, 'reward': 0.8518542131594571, 'waypoint': 'left'}
Agent previous state: ('left', 'red', 'left', None, None)
Agent properly idled at a red light. (rewarded 0.85)
37% of time remaining to reach destination.
/-------------------
| Step 19 Results
\-------------------
Environment.step(): t = 19
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 1.30396827826
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'right'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'right', 'left', None), 'deadline': 11, 't': 19, 'action': 'right', 'reward': 1.3039682782569537, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'right', 'left', None)
Agent drove right instead of left. (rewarded 1.30)
33% of time remaining to reach destination.
/-------------------
| Step 20 Results
\-------------------
Environment.step(): t = 20
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 1.16718228019
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 10, 't': 20, 'action': 'right', 'reward': 1.1671822801859904, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.17)
30% of time remaining to reach destination.
/-------------------
| Step 21 Results
\-------------------
Environment.step(): t = 21
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 1.62406692777
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'right'), 'deadline': 9, 't': 21, 'action': 'right', 'reward': 1.6240669277715158, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'right')
Agent followed the waypoint right. (rewarded 1.62)
27% of time remaining to reach destination.
/-------------------
| Step 22 Results
\-------------------
Environment.step(): t = 22
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: right, reward: 1.59939066244
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 8, 't': 22, 'action': 'right', 'reward': 1.599390662437709, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 1.60)
23% of time remaining to reach destination.
/-------------------
| Step 23 Results
\-------------------
Environment.step(): t = 23
Environment.act() [POST]: location: (6, 2), heading: (0, 1), action: right, reward: 1.15270753405
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': 'left', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, 'left', 'forward'), 'deadline': 7, 't': 23, 'action': 'right', 'reward': 1.1527075340473791, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, 'left', 'forward')
Agent drove right instead of left. (rewarded 1.15)
20% of time remaining to reach destination.
/-------------------
| Step 24 Results
\-------------------
Environment.step(): t = 24
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: right, reward: 0.807861790106
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, None, None), 'deadline': 6, 't': 24, 'action': 'right', 'reward': 0.8078617901064715, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, None, None)
Agent followed the waypoint right. (rewarded 0.81)
17% of time remaining to reach destination.
/-------------------
| Step 25 Results
\-------------------
Environment.step(): t = 25
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 0.964976275665
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', 'forward', None, None), 'deadline': 5, 't': 25, 'action': None, 'reward': 0.964976275665351, 'waypoint': 'right'}
Agent previous state: ('right', 'red', 'forward', None, None)
Agent properly idled at a red light. (rewarded 0.96)
13% of time remaining to reach destination.
/-------------------
| Step 26 Results
\-------------------
Environment.step(): t = 26
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: None, reward: 0.314491735837
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, 'forward'), 'deadline': 4, 't': 26, 'action': None, 'reward': 0.314491735836691, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.31)
10% of time remaining to reach destination.
/-------------------
| Step 27 Results
\-------------------
Environment.step(): t = 27
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 1.32299622366
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': None, 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', None, 'forward'), 'deadline': 3, 't': 27, 'action': 'right', 'reward': 1.3229962236569082, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', None, 'forward')
Agent followed the waypoint right. (rewarded 1.32)
7% of time remaining to reach destination.
/-------------------
| Step 28 Results
\-------------------
Environment.step(): t = 28
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: right, reward: 0.36972689846
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('right', 'red', None, None, None), 'deadline': 2, 't': 28, 'action': 'right', 'reward': 0.3697268984595412, 'waypoint': 'right'}
Agent previous state: ('right', 'red', None, None, None)
Agent followed the waypoint right. (rewarded 0.37)
3% of time remaining to reach destination.
/-------------------
| Step 29 Results
\-------------------
Environment.step(): t = 29
Environment.act() [POST]: location: (6, 7), heading: (1, 0), action: None, reward: 1.82330103071
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, None), 'deadline': 1, 't': 29, 'action': None, 'reward': 1.8233010307145077, 'waypoint': 'left'}
Environment.step(): Primary agent ran out of time! Trial aborted.
Agent previous state: ('left', 'red', None, None, None)
Agent properly idled at a red light. (rewarded 1.82)
0% of time remaining to reach destination.
Trial Aborted!
Agent did not reach the destination.
/-------------------------
| Testing trial 9
\-------------------------
Environment.reset(): Trial set up with start = (8, 2), destination = (5, 6), deadline = 25
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: right, reward: 2.46339549411
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'right', 'left': 'left'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'left', 'right', None), 'deadline': 25, 't': 0, 'action': 'right', 'reward': 2.463395494112828, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'left', 'right', None)
Agent followed the waypoint right. (rewarded 2.46)
96% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.18285767614
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'left'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'left', None, 'left'), 'deadline': 24, 't': 1, 'action': None, 'reward': 2.182857676144282, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'left', None, 'left')
Agent properly idled at a red light. (rewarded 2.18)
92% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 1.13969957554
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 23, 't': 2, 'action': None, 'reward': 1.1396995755373087, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 1.14)
88% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.91989021964
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, None, 'left'), 'deadline': 22, 't': 3, 'action': None, 'reward': 2.9198902196438494, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, None, 'left')
Agent properly idled at a red light. (rewarded 2.92)
84% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 1.93828418281
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': 'right', 'left': None}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', None, 'right', 'left'), 'deadline': 21, 't': 4, 'action': None, 'reward': 1.9382841828090582, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', None, 'right', 'left')
Agent properly idled at a red light. (rewarded 1.94)
80% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (7, 2), heading: (-1, 0), action: None, reward: 2.14486633071
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 20, 't': 5, 'action': None, 'reward': 2.1448663307050166, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 2.14)
76% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (6, 2), heading: (-1, 0), action: forward, reward: 1.72796273155
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 19, 't': 6, 'action': 'forward', 'reward': 1.7279627315486528, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.73)
72% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (5, 2), heading: (-1, 0), action: forward, reward: 2.83935279872
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'forward', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', 'forward', 'forward', None), 'deadline': 18, 't': 7, 'action': 'forward', 'reward': 2.839352798723154, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', 'forward', 'forward', None)
Agent followed the waypoint forward. (rewarded 2.84)
68% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act() [POST]: location: (5, 7), heading: (0, -1), action: right, reward: 1.04248094457
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', None, 'forward', 'left'), 'deadline': 17, 't': 8, 'action': 'right', 'reward': 1.0424809445655607, 'waypoint': 'right'}
Agent previous state: ('right', 'green', None, 'forward', 'left')
Agent followed the waypoint right. (rewarded 1.04)
64% of time remaining to reach destination.
/-------------------
| Step 9 Results
\-------------------
Environment.step(): t = 9
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (5, 6), heading: (0, -1), action: forward, reward: 2.68398456264
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, None), 'deadline': 16, 't': 9, 'action': 'forward', 'reward': 2.683984562638213, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, None)
Agent followed the waypoint forward. (rewarded 2.68)
60% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
/-------------------------
| Testing trial 10
\-------------------------
Environment.reset(): Trial set up with start = (5, 7), destination = (7, 3), deadline = 20
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
Simulating trial. . .
epsilon = 0.0000; alpha = 0.0000
/-------------------
| Step 0 Results
\-------------------
Environment.step(): t = 0
Environment.act() [POST]: location: (4, 7), heading: (-1, 0), action: left, reward: 1.6813316997
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('right', 'green', 'forward', 'left', 'left'), 'deadline': 20, 't': 0, 'action': 'left', 'reward': 1.6813316997024574, 'waypoint': 'right'}
Agent previous state: ('right', 'green', 'forward', 'left', 'left')
Agent drove left instead of right. (rewarded 1.68)
95% of time remaining to reach destination.
/-------------------
| Step 1 Results
\-------------------
Environment.step(): t = 1
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: left, reward: 1.17711297031
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': 'left', 'left': 'forward'}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', 'forward', 'left', None), 'deadline': 19, 't': 1, 'action': 'left', 'reward': 1.1771129703078198, 'waypoint': 'left'}
Agent previous state: ('left', 'green', 'forward', 'left', None)
Agent followed the waypoint left. (rewarded 1.18)
90% of time remaining to reach destination.
/-------------------
| Step 2 Results
\-------------------
Environment.step(): t = 2
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 2.36864105865
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'right'), 'deadline': 18, 't': 2, 'action': None, 'reward': 2.368641058646424, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'right')
Agent properly idled at a red light. (rewarded 2.37)
85% of time remaining to reach destination.
/-------------------
| Step 3 Results
\-------------------
Environment.step(): t = 3
Environment.act() [POST]: location: (4, 2), heading: (0, 1), action: None, reward: 0.966244789148
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'forward', 'right': None, 'left': None}, 'violation': 0, 'light': 'red', 'state': ('left', 'red', None, None, 'forward'), 'deadline': 17, 't': 3, 'action': None, 'reward': 0.9662447891478716, 'waypoint': 'left'}
Agent previous state: ('left', 'red', None, None, 'forward')
Agent properly idled at a red light. (rewarded 0.97)
80% of time remaining to reach destination.
/-------------------
| Step 4 Results
\-------------------
Environment.step(): t = 4
Environment.act() [POST]: location: (4, 3), heading: (0, 1), action: forward, reward: 1.65636094878
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'forward', 'right': 'forward', 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, 'forward', 'forward'), 'deadline': 16, 't': 4, 'action': 'forward', 'reward': 1.656360948781106, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, 'forward', 'forward')
Agent drove forward instead of left. (rewarded 1.66)
75% of time remaining to reach destination.
/-------------------
| Step 5 Results
\-------------------
Environment.step(): t = 5
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: left, reward: 2.82767571897
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': None, 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('left', 'green', None, None, None), 'deadline': 15, 't': 5, 'action': 'left', 'reward': 2.8276757189733357, 'waypoint': 'left'}
Agent previous state: ('left', 'green', None, None, None)
Agent followed the waypoint left. (rewarded 2.83)
70% of time remaining to reach destination.
/-------------------
| Step 6 Results
\-------------------
Environment.step(): t = 6
Environment.act() [POST]: location: (5, 3), heading: (1, 0), action: None, reward: 2.14982601252
Environment.act(): Step data: {'inputs': {'light': 'red', 'oncoming': 'left', 'right': None, 'left': 'right'}, 'violation': 0, 'light': 'red', 'state': ('forward', 'red', 'right', None, 'left'), 'deadline': 14, 't': 6, 'action': None, 'reward': 2.149826012518226, 'waypoint': 'forward'}
Agent previous state: ('forward', 'red', 'right', None, 'left')
Agent properly idled at a red light. (rewarded 2.15)
65% of time remaining to reach destination.
/-------------------
| Step 7 Results
\-------------------
Environment.step(): t = 7
Environment.act() [POST]: location: (6, 3), heading: (1, 0), action: forward, reward: 1.71864385367
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'left', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'left'), 'deadline': 13, 't': 7, 'action': 'forward', 'reward': 1.7186438536656101, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'left')
Agent followed the waypoint forward. (rewarded 1.72)
60% of time remaining to reach destination.
/-------------------
| Step 8 Results
\-------------------
Environment.step(): t = 8
Environment.act(): Primary agent has reached destination!
Environment.act() [POST]: location: (7, 3), heading: (1, 0), action: forward, reward: 2.4269943445
Environment.act(): Step data: {'inputs': {'light': 'green', 'oncoming': 'right', 'right': None, 'left': None}, 'violation': 0, 'light': 'green', 'state': ('forward', 'green', None, None, 'right'), 'deadline': 12, 't': 8, 'action': 'forward', 'reward': 2.426994344502333, 'waypoint': 'forward'}
Agent previous state: ('forward', 'green', None, None, 'right')
Agent followed the waypoint forward. (rewarded 2.43)
55% of time remaining to reach destination.
Trial Completed!
Agent reached the destination.
Simulation ended. . .
In [31]:
# self.epsilon = math.exp(-(self.a*self.t))
# epsilon-tolerance = 0.05 alpha = 0.5
# 训练次数：300
vs.plot_trials('sim_improved-learning.csv')
问题7¶
利用上面的从你改进的Q-Learning模拟中得到的可视化结果，像在问题6那样，给出关于改进的驾驶代理程序的最终分析和观察。你需要回答的问题：
- 使用了什么epsilon（探索因子）的衰减函数？
- 在测试之前驾驶代理大约需要做多少训练试验？
- 你用了什么epsilon-tolerance和alpha（学习率）值？为什么？
- 与之前的默认Q-Learning学习器相比，这个Q-Learning学习器有多少改进？
- 你会说Q-Learning学习器的结果表明了你的驾驶代理程序成功地学习了一个合适的策略吗？
- 你对智能出租车的安全性和可靠性评分满意吗？
回答：
- 使用了epsilon的衰减函数如下： $$ \epsilon = e^{-at}, \textrm{for } 0 < a < 1 \hspace{50px} $$
- 大约需要做300次训练试验
- epsilon-tolerance为0.05，alpha（学习率）值为0.5。epsilon-tolerance越小训练次数越多时间越长，而学习率越高每次动作获得的奖励对形成更好的策略贡献越多，但同时短期奖励对策略影响越大，因此适当提升学习率并增大epsilon-tolerance，提高学习效率并降低训练时间成本同时获得更好的策略。
- 改进很多，默认的学习器并不能从动作的奖励中获得反馈，从经验中学习，而这个Q-learning学习器能够综合考虑过去的“成功经验”和“当前奖励”做出决策，并以epsilon的概率去搜索更多未知策略，避免陷入局部最优，是一个智能的学习程序。
- 是的
- 对于一个小程序，还算满意，但仍有很多改进空间。这只是一个十分简化的智能车学习器，真实环境中需要考虑的影响因素远远多于这些，交通灯信号更复杂、交通法规中的各种优先规则以及路面指示线指示牌都是需要包含在模型中的参数。如果真正想要上路行驶，那么对可靠性的要求也十分严苛，应尽量达到100%。
定义一个最优策略¶
有时，对于重要的问题“我要让我的代理程序学习什么？”的答案，只是理论性的，无法具体描述。然而这里，你可以具体定义代理程序要学什么，就是美国通行权交通法案。这些法律是已知信息，你可以基于这些法律，为智能出租车进一步定义每一个状态所做的最优动作。在那种情况下，我们称这一系列最优状态-动作配对为最优策略。因此，不像那些理论性的回答，不仅通过收到的奖励（惩罚），而且纯观察，代理程序是否在“错误”地行动能很清晰地得知。如果代理程序闯了红灯，我们既看见它获得了一个负面奖励，也知道这是一个错误的行为。这可以用来帮你验证驾驶代理程序习得的策略是否正确，或只是个次优策略。
问题 8¶
给出几个关于最优策略是什么样子的例子（用你已定义的状态）。之后，查看'sim_improved-learning.txt'文本文件，看你的改进的Q-Learning算法的结果。每个从模拟中纪录的状态，对于给定的状态，策略（得分最高的动作）是否正确？是否对于有些状态，有策略不同于预期的最优策略？给出一个状态和记录的状态-动作的奖励，解释为什么正是个正确的策略。
回答：
状态由5个特征描述(waypoint, light, left, right, oncoming)，当状态为(left, red, None,None,'forward')时最优策略是不动；当状态为(right, red, None, 'left', 'right')最优策略是不动；当状态为(Forward, Green, None,None,None)时最优策略是直行。
- ('left', 'red', None, None, 'forward')¶
-- forward : -10.56 -- right : 0.94 -- None : 1.80 -- left : -9.50 （策略正确）
- ('right', 'red', None, 'left', 'right')¶
-- forward : -4.53 -- right : 0.00 -- None : 1.12 -- left : 0.00 （策略正确）
- ('forward', 'green', 'forward', 'left', 'left')¶
-- forward : 0.00 -- right : 0.00 -- None : 0.00 -- left : 0.63 （策略错误： 应该等前方的车左转结束后前行，而策略给出左转，可能会导致偏离最佳路线，不能按时到达终点。最佳策略应选择不动。）
选做：未来奖励 - 折扣因子 'gamma'¶
也许你会好奇，作为Q-Learning算法的一部分，之前要求你在实现中不要使用折扣引子'gamma'。在算法中包含未来奖励能有助于在未来状态回溯到当前状态时的反向正面奖励。本质上，如果给予驾驶代理程序执行若干动作到达不同状态的选择，包含未来奖励会是代理程序偏向可以得到更多奖励的状态。一个例子是驶向目的的驾驶代理程序：所有行动和奖励都相等，那么理论上如果到达目的地会有额外奖励，驶向目的会获得更好的奖励。然而，即使在这个项目里，驾驶代理程序也要在规定的时间里到达目的地，包含未来奖励不会有益于代理程序。实际上，如果代理程序给予多次试验学习，它甚至会给Q-value带来负面影响！
可选问题 9¶
在项目中有两个特点使得未来奖励在这个Q-Learning算法无效。一个特点是关于智能出租车本身，另一个是关于环境。你能指出它们是什么以及为什么未来奖励不会在这个项目中起效？
回答：
注意:当你写完了所有的代码，并且回答了所有的问题。你就可以把你的 iPython Notebook 导出成 HTML 文件。你可以在菜单栏，这样导出File -> Download as -> HTML (.html)把这个 HTML 和这个 iPython notebook 一起做为你的作业提交。